Changes between Initial Version and Version 1 of ESGFNodeFullAutoInstall


Ignore:
Timestamp:
Jan 10, 2018 11:35:28 AM (5 years ago)
Author:
zequi
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ESGFNodeFullAutoInstall

    v1 v1  
     1= ESGF Local Node Deployment Tutorial =
     2
     3This page shows how to deploy an ESGF Node that provides data, index, compute and idp services and belongs to the esgf-test federation. The purpose of this node is to test publication and other processes before applying them into production.
     4
     5UNICAN belongs to the Tier 2 group of nodes so the production node MUST be data only.
     6
     7This page assumes that command are executed by the root user (or sudo -s).
     8
     9== Index ==
     100. Prerequisites
     111. Previous installation clean up
     122. Installation from scratch using autoinstaller
     133. Configuration for publication
     144. Publish the test dataset
     155. Publish CORDEX datasets
     166. Known issues
     177. References
     18
     19== Prerequisites ==
     20
     211. You have to create a globus account - https://www.globusid.org/create
     222. Open ports: [https://meteo.unican.es/trac/wiki/ESGFNodeInstallation Required open ports]
     23
     24{{{
     25[root@spock ~]# iptables -L
     26Chain INPUT (policy DROP)
     27target     prot opt source               destination         
     28ACCEPT     all  --  anywhere             anywhere           
     29ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
     30ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:http state NEW
     31ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:https state NEW
     32ACCEPT     tcp  --  192.168.202.0/24     anywhere            tcp dpt:ssh
     33ACCEPT     tcp  --  193.144.202.0/24     anywhere            tcp dpt:ssh
     34ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:gsiftp
     35ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:7512
     36ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpts:50000:51000
     37ACCEPT     icmp --  192.168.202.0/24     anywhere           
     38ACCEPT     icmp --  anywhere             anywhere            icmp echo-request state NEW,RELATED,ESTABLISHED
     39
     40Chain FORWARD (policy DROP)
     41target     prot opt source               destination         
     42
     43Chain OUTPUT (policy ACCEPT)
     44target     prot opt source               destination         
     45
     46Chain SSHSCAN (0 references)
     47target     prot opt source               destination         
     48}}}
     49
     50== 1. Previous installation clean up ==
     51
     52Execute {{{esg-node stop}}} in order to stop the current ESGF services (in case they are running).
     53
     54{{{
     55[root@spock ~]# esg-node stop
     56
     57
     58  EEEEEEEEEEEEEEEEEEEEEE   SSSSSSSSSSSSSSS         GGGGGGGGGGGGGFFFFFFFFFFFFFFFFFFFFFF
     59  E::::::::::::::::::::E SS:::::::::::::::S     GGG::::::::::::GF::::::::::::::::::::F
     60  E::::::::::::::::::::ES:::::SSSSSS::::::S   GG:::::::::::::::GF::::::::::::::::::::F
     61  EE::::::EEEEEEEEE::::ES:::::S     SSSSSSS  G:::::GGGGGGGG::::GFF::::::FFFFFFFFF::::F
     62    E:::::E       EEEEEES:::::S             G:::::G       GGGGGG  F:::::F       FFFFFF
     63    E:::::E             S:::::S            G:::::G                F:::::F
     64    E::::::EEEEEEEEEE    S::::SSSS         G:::::G                F::::::FFFFFFFFFF
     65    E:::::::::::::::E     SS::::::SSSSS    G:::::G    GGGGGGGGGG  F:::::::::::::::F
     66    E:::::::::::::::E       SSS::::::::SS  G:::::G    G::::::::G  F:::::::::::::::F
     67    E::::::EEEEEEEEEE          SSSSSS::::S G:::::G    GGGGG::::G  F::::::FFFFFFFFFF
     68    E:::::E                         S:::::SG:::::G        G::::G  F:::::F
     69    E:::::E       EEEEEE            S:::::S G:::::G       G::::G  F:::::F
     70  EE::::::EEEEEEEE:::::ESSSSSSS     S:::::S  G:::::GGGGGGGG::::GFF:::::::FF
     71  E::::::::::::::::::::ES::::::SSSSSS:::::S   GG:::::::::::::::GF::::::::FF
     72  E::::::::::::::::::::ES:::::::::::::::SS      GGG::::::GGG:::GF::::::::FF
     73  EEEEEEEEEEEEEEEEEEEEEE SSSSSSSSSSSSSSS           GGGGGG   GGGGFFFFFFFFFFF.llnl.gov
     74
     75Checking that you have root privs on spock.meteo.unican.es... [OK]
     76Checking requisites...
     77
     78Using IP: 193.144.184.40
     79Stopping Globus Services for Data-Node... (GridFTP) stop_globus_services for datanode
     80Stopping globus-gridftp-server: OK
     81Stopping dashboard provider...
     82Shutting down ESGF Information Provider... [OK]
     83Shutting down :  esgfnmdNode Manager process not present. Attempting to wipe  PIDFILE and LOCKFILE.
     84Tomcat (jsvc) process is running...
     85
     86stop tomcat: /usr/local/tomcat/bin/jsvc -pidfile /var/run/tomcat-jsvc.pid -stop org.apache.catalina.startup.Bootstrap
     87(please wait)
     885 Z tomcat    7111     1  0  80   0 -     0 do_exi  2017 ?        00:24:39 [jsvc] <defunct>
     89postmaster (pid  30159) is running...
     90Stopping postgresql service:                               [  OK  ]
     91Stopping httpd:                                            [  OK  ]
     92Running shutdown hooks...
     93
     94---------------------------
     95Running Node Services...
     96node type: [ data ] (4)
     97---------------------------
     98---------------------------
     99}}}
     100
     101Execute {{{source /usr/local/bin/esg-purge.sh && esg-purge all}}}
     102
     103== Installation from scratch using autoinstaller ==
     104
     105The autoinstaller allows the installation of data only and data+idp+compute+index nodes. Any other combinations should be installed manually. See https://github.com/ESGF/esgf-installer/wiki/ESGF-Installation-Using-Autoinstaller.
     106
     107As root (not sudo!):
     108{{{
     109cd /usr/local/bin
     110wget -O esg-bootstrap http://distrib-coffee.ipsl.jussieu.fr/pub/esgf/dist/esgf-installer/2.5/esg-bootstrap --no-check-certificate 
     111chmod 555 esg-bootstrap
     112./esg-bootstrap
     113}}}
     114
     115Edit /usr/local/etc/esg-autoinstall.conf. It should looks like:
     116{{{
     117#!/usr/bin/expect -f
     118# -*- mode:shell-script -*-
     119#
     120# Configuration file for the esg-autoinstall Expect script.
     121#
     122# If you are opening this file as 'esg-autoinstall.template', either
     123# from /usr/local/etc or from a Git repository, DO NOT MODIFY IT
     124# DIRECTLY.  It must be copied to /usr/local/etc/esg-autoinstall.conf
     125# before it will be read.
     126#
     127# THIS FILE CONTAINS PASSWORDS -- ensure that when you copy it you set
     128# the permissions to 600 and the ownership to root, e.g.
     129#   chown root /usr/local/etc/esg-autoinstall.conf
     130#   chmod 600 /usr/local/etc/esg-autoinstall.conf
     131#
     132
     133### Local Node Settings ###
     134
     135# Set this to "data" or "data compute" for a data-only node
     136# Set this to "index" for an index-only node
     137set NODETYPE "index idp data compute"
     138
     139# Optional install flags
     140# Possibilites include "--devel" or "--debug"
     141set INSTALLFLAGS ""
     142
     143# Fully qualified domain name of the node
     144set FQDN "spock.meteo.unican.es"
     145
     146# Shortname of the node
     147set SHORTNAME "unican test"
     148
     149# Long descriptive name of the node
     150set LONGNAME "unican test"
     151
     152# Admin password (alphanumeric-only)
     153set ADMINPASS "PASSWORD"
     154
     155# IP address cleared for admin-restricted pages
     156set ADMINIP "0.0.0.0"
     157
     158# Password for low-privilege publisher database account (alphanumeric-only)
     159set DBLOWPASS "PASSWORD"
     160
     161# Globus Online username and password
     162# (you must have set up this account in advance)
     163# If your password contains URL special characters, installation may fail.
     164set GLOBUSUSER "YOUR-GLOBUS-USER"
     165set GLOBUSPASS "YOUR-GLOBUS-PASSWORD"
     166
     167# Organization name
     168set ORGNAME "unican"
     169
     170# Namespace (reverse FQDN, i.e. "gov.llnl")
     171set NAMESPACE "es.unican.meteo.spock"
     172
     173# Peer Group (valid values are usually "esgf-test" and "esgf-prod")
     174set PEERGROUP "esgf-test"
     175
     176# Default Peer (set this to empty for the default peergroup member, or to your own FQDN)
     177set DEFAULTPEER "spock.meteo.unican.es"
     178
     179# Hostname of the node to publish to
     180set PUBLISHNODE "spock.meteo.unican.es"
     181
     182# Use an external IDP Peer?
     183# (Unless you know you need to run your own, put y)
     184set EXTERNALIDP "n"
     185
     186# FQDN of the external IDP Peer
     187set IDPPEER "spock.meteo.unican.es"
     188
     189# E-mail address notifications will be sent as
     190set ADMINEMAIL "ezequiel.cimadevilla@unican.es"
     191
     192# Default answer for installing Globus
     193# Should be be N for upgrade and Y for fresh install
     194set GLOBUS "y"
     195
     196# Default answer for automatic peering
     197# Should be be N for upgrade and Y for fresh install
     198set AUTOMATICPEER "y"
     199
     200set THREDDS "N"
     201
     202### Special-use variables (you probably don't want to change these) ###
     203
     204# Which install script to run
     205set ESGNODESCRIPT /usr/local/bin/esg-node
     206
     207# Disable timeouts (the installer can take a long time)
     208set timeout -1
     209
     210# Database connection string
     211# (form: [username]@[host]:[port]/[database])
     212# Note: all elements are mandatory, and the "postgresql://" header is
     213# automatically prepended by the esg-node script!
     214# Currently, the only allowed database name is "esgcet"
     215## DO NOT CURRENTLY SET THIS TO ANYTHING BUT ""
     216#set DBSTRING "esg@localhost:5432/esgcet
     217set DBSTRING ""
     218
     219# Is this an externally managed database?
     220# (if DBSTRING isn't pointed at localhost, the answer is yes)
     221# DO NOT CURRENTLY SET THIS TO YES
     222set DBEXTERNAL no
     223
     224# Low-privilege database account (this may need to be "esgcet")
     225set DBLOWUSER "esgcet"
     226
     227# PostgreSQL port number
     228# DO NOT CURRENTLY CHANGE THIS
     229set PGPORT 5432
     230
     231# Force recreation of SOLR replica indexes
     232set SOLRREINDEX ""
     233}}}