Changes between Initial Version and Version 1 of ESGFNodeFullAutoInstall

Timestamp:

Jan 10, 2018 11:35:28 AM (5 years ago)

Author:

zequi

Comment:

--

ESGFNodeFullAutoInstall

@@ +1 @@
+= ESGF Local Node Deployment Tutorial =
+
+This page shows how to deploy an ESGF Node that provides data, index, compute and idp services and belongs to the esgf-test federation. The purpose of this node is to test publication and other processes before applying them into production.
+
+UNICAN belongs to the Tier 2 group of nodes so the production node MUST be data only.
+
+This page assumes that command are executed by the root user (or sudo -s).
+
+== Index ==
+0. Prerequisites
+1. Previous installation clean up
+2. Installation from scratch using autoinstaller
+3. Configuration for publication
+4. Publish the test dataset
+5. Publish CORDEX datasets
+6. Known issues
+7. References
+
+== Prerequisites ==
+
+1. You have to create a globus account - https://www.globusid.org/create
+2. Open ports: [https://meteo.unican.es/trac/wiki/ESGFNodeInstallation Required open ports]
+
+{{{
+[root@spock ~]# iptables -L
+Chain INPUT (policy DROP)
+target     prot opt source               destination         
+ACCEPT     all  --  anywhere             anywhere            
+ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED 
+ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:http state NEW 
+ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https state NEW 
+ACCEPT     tcp  --  192.168.202.0/24     anywhere            tcp dpt:ssh 
+ACCEPT     tcp  --  193.144.202.0/24     anywhere            tcp dpt:ssh 
+ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:gsiftp 
+ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:7512 
+ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpts:50000:51000 
+ACCEPT     icmp --  192.168.202.0/24     anywhere            
+ACCEPT     icmp --  anywhere             anywhere            icmp echo-request state NEW,RELATED,ESTABLISHED 
+
+Chain FORWARD (policy DROP)
+target     prot opt source               destination         
+
+Chain OUTPUT (policy ACCEPT)
+target     prot opt source               destination         
+
+Chain SSHSCAN (0 references)
+target     prot opt source               destination         
+}}}
+
+== 1. Previous installation clean up ==
+
+Execute {{{esg-node stop}}} in order to stop the current ESGF services (in case they are running).
+
+{{{
+[root@spock ~]# esg-node stop
+
+
+  EEEEEEEEEEEEEEEEEEEEEE   SSSSSSSSSSSSSSS         GGGGGGGGGGGGGFFFFFFFFFFFFFFFFFFFFFF
+  E::::::::::::::::::::E SS:::::::::::::::S     GGG::::::::::::GF::::::::::::::::::::F
+  E::::::::::::::::::::ES:::::SSSSSS::::::S   GG:::::::::::::::GF::::::::::::::::::::F
+  EE::::::EEEEEEEEE::::ES:::::S     SSSSSSS  G:::::GGGGGGGG::::GFF::::::FFFFFFFFF::::F
+    E:::::E       EEEEEES:::::S             G:::::G       GGGGGG  F:::::F       FFFFFF
+    E:::::E             S:::::S            G:::::G                F:::::F
+    E::::::EEEEEEEEEE    S::::SSSS         G:::::G                F::::::FFFFFFFFFF
+    E:::::::::::::::E     SS::::::SSSSS    G:::::G    GGGGGGGGGG  F:::::::::::::::F
+    E:::::::::::::::E       SSS::::::::SS  G:::::G    G::::::::G  F:::::::::::::::F
+    E::::::EEEEEEEEEE          SSSSSS::::S G:::::G    GGGGG::::G  F::::::FFFFFFFFFF
+    E:::::E                         S:::::SG:::::G        G::::G  F:::::F
+    E:::::E       EEEEEE            S:::::S G:::::G       G::::G  F:::::F
+  EE::::::EEEEEEEE:::::ESSSSSSS     S:::::S  G:::::GGGGGGGG::::GFF:::::::FF
+  E::::::::::::::::::::ES::::::SSSSSS:::::S   GG:::::::::::::::GF::::::::FF
+  E::::::::::::::::::::ES:::::::::::::::SS      GGG::::::GGG:::GF::::::::FF
+  EEEEEEEEEEEEEEEEEEEEEE SSSSSSSSSSSSSSS           GGGGGG   GGGGFFFFFFFFFFF.llnl.gov
+
+Checking that you have root privs on spock.meteo.unican.es... [OK]
+Checking requisites... 
+
+Using IP: 193.144.184.40
+Stopping Globus Services for Data-Node... (GridFTP) stop_globus_services for datanode
+Stopping globus-gridftp-server: OK
+Stopping dashboard provider...
+Shutting down ESGF Information Provider... [OK]
+Shutting down :  esgfnmdNode Manager process not present. Attempting to wipe  PIDFILE and LOCKFILE.
+Tomcat (jsvc) process is running... 
+
+stop tomcat: /usr/local/tomcat/bin/jsvc -pidfile /var/run/tomcat-jsvc.pid -stop org.apache.catalina.startup.Bootstrap
+(please wait)
+5 Z tomcat    7111     1  0  80   0 -     0 do_exi  2017 ?        00:24:39 [jsvc] <defunct>
+postmaster (pid  30159) is running...
+Stopping postgresql service:                               [  OK  ]
+Stopping httpd:                                            [  OK  ]
+Running shutdown hooks...
+
+---------------------------
+Running Node Services... 
+node type: [ data ] (4) 
+---------------------------
+---------------------------
+}}}
+
+Execute {{{source /usr/local/bin/esg-purge.sh && esg-purge all}}}
+
+== Installation from scratch using autoinstaller ==
+
+The autoinstaller allows the installation of data only and data+idp+compute+index nodes. Any other combinations should be installed manually. See https://github.com/ESGF/esgf-installer/wiki/ESGF-Installation-Using-Autoinstaller.
+
+As root (not sudo!):
+{{{
+cd /usr/local/bin
+wget -O esg-bootstrap http://distrib-coffee.ipsl.jussieu.fr/pub/esgf/dist/esgf-installer/2.5/esg-bootstrap --no-check-certificate  
+chmod 555 esg-bootstrap
+./esg-bootstrap
+}}}
+
+Edit /usr/local/etc/esg-autoinstall.conf. It should looks like:
+{{{
+#!/usr/bin/expect -f
+# -*- mode:shell-script -*-
+#
+# Configuration file for the esg-autoinstall Expect script.
+#
+# If you are opening this file as 'esg-autoinstall.template', either
+# from /usr/local/etc or from a Git repository, DO NOT MODIFY IT
+# DIRECTLY.  It must be copied to /usr/local/etc/esg-autoinstall.conf
+# before it will be read.
+#
+# THIS FILE CONTAINS PASSWORDS -- ensure that when you copy it you set
+# the permissions to 600 and the ownership to root, e.g.
+#   chown root /usr/local/etc/esg-autoinstall.conf
+#   chmod 600 /usr/local/etc/esg-autoinstall.conf
+#
+
+### Local Node Settings ###
+
+# Set this to "data" or "data compute" for a data-only node
+# Set this to "index" for an index-only node
+set NODETYPE "index idp data compute"
+
+# Optional install flags
+# Possibilites include "--devel" or "--debug"
+set INSTALLFLAGS ""
+
+# Fully qualified domain name of the node
+set FQDN "spock.meteo.unican.es"
+
+# Shortname of the node
+set SHORTNAME "unican test"
+
+# Long descriptive name of the node
+set LONGNAME "unican test"
+
+# Admin password (alphanumeric-only)
+set ADMINPASS "PASSWORD"
+
+# IP address cleared for admin-restricted pages
+set ADMINIP "0.0.0.0"
+
+# Password for low-privilege publisher database account (alphanumeric-only)
+set DBLOWPASS "PASSWORD"
+
+# Globus Online username and password
+# (you must have set up this account in advance)
+# If your password contains URL special characters, installation may fail.
+set GLOBUSUSER "YOUR-GLOBUS-USER"
+set GLOBUSPASS "YOUR-GLOBUS-PASSWORD"
+
+# Organization name
+set ORGNAME "unican"
+
+# Namespace (reverse FQDN, i.e. "gov.llnl")
+set NAMESPACE "es.unican.meteo.spock"
+
+# Peer Group (valid values are usually "esgf-test" and "esgf-prod")
+set PEERGROUP "esgf-test"
+
+# Default Peer (set this to empty for the default peergroup member, or to your own FQDN)
+set DEFAULTPEER "spock.meteo.unican.es"
+
+# Hostname of the node to publish to
+set PUBLISHNODE "spock.meteo.unican.es"
+
+# Use an external IDP Peer?
+# (Unless you know you need to run your own, put y)
+set EXTERNALIDP "n"
+
+# FQDN of the external IDP Peer
+set IDPPEER "spock.meteo.unican.es"
+
+# E-mail address notifications will be sent as
+set ADMINEMAIL "ezequiel.cimadevilla@unican.es"
+
+# Default answer for installing Globus
+# Should be be N for upgrade and Y for fresh install
+set GLOBUS "y"
+
+# Default answer for automatic peering
+# Should be be N for upgrade and Y for fresh install
+set AUTOMATICPEER "y"
+
+set THREDDS "N"
+
+### Special-use variables (you probably don't want to change these) ###
+
+# Which install script to run
+set ESGNODESCRIPT /usr/local/bin/esg-node
+
+# Disable timeouts (the installer can take a long time)
+set timeout -1
+
+# Database connection string
+# (form: [username]@[host]:[port]/[database])
+# Note: all elements are mandatory, and the "postgresql://" header is
+# automatically prepended by the esg-node script!
+# Currently, the only allowed database name is "esgcet"
+## DO NOT CURRENTLY SET THIS TO ANYTHING BUT ""
+#set DBSTRING "esg@localhost:5432/esgcet
+set DBSTRING ""
+
+# Is this an externally managed database?
+# (if DBSTRING isn't pointed at localhost, the answer is yes)
+# DO NOT CURRENTLY SET THIS TO YES
+set DBEXTERNAL no
+
+# Low-privilege database account (this may need to be "esgcet")
+set DBLOWUSER "esgcet"
+
+# PostgreSQL port number
+# DO NOT CURRENTLY CHANGE THIS
+set PGPORT 5432
+
+# Force recreation of SOLR replica indexes
+set SOLRREINDEX ""
+}}}