Version 13 (modified by carlos, 8 years ago) (diff)


This installation guide will provide instructions about how to install an ESGF data/compute node. For instance, a VM with 1 core, 2GB of RAM and 20GB of Disk. For the installation process it's recommended to provide more cores than 1 core.


TCP and UDP ports firewall configuration [[FootNote(...)]]

Corporate Firewall

Port Direction Type Application Description
80 in tcp Tomcat Web server access
443 in tcp Tomcat SSL - Secure Web Server Access.
5432 in tcp Postgres Postgres Access. (not external: by default bound ONLY TO LOCAL INTERFACE)
2811 in tcp GridFTP user-configured GridFTP Server control channel
(60000-61000) in/out tcp GridFTP user-configured GridFTP Server data channel (or as defined in the global variable GLOBUS_TCP_PORT_RANGE)
2812 in tcp GridFTP BDM-configured GridFTP Server control channel. May run together with the user-configured one though not recommended - system resource intensive!
(60000-61000) in/out tcp GridFTP BDM-configured GridFTP Server data channel. May run together with the user-configured one though not recommended - system resource intensive!
7512 out tcp MyProxy MyProxy client access to the certificate repository
8984 - tcp esgf-search (Tomcat)local connection to the Solr master instance (not external!)
8983 in/out tcp esgf-search (Tomcat) Connection to remotes Solr slave instance. Used in distributed search (shard).
80 out tcp esg-publisher Local connection to THREDDS server (e.g., to check catalogs) and other nodes (node-manager)
443 out tcp esg-publisher Local secure connection to THREDDS server (e.g., to restart the application) and to the idp

IPTables configuration

Add these rules to the IPTables configuration file, i.e. /etc/sysconfig/iptables

-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 2811 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 2812 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8984 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 8983 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 60000:61000 -j ACCEPT

and restart IPTables services

$ services iptables restart

Install RPM packages

Install the sourceforge RPM repository for the *ExtUtils* packages:

$ rpm -i -U -h

And now the ESGF required RPM packages

$ yum install autoconf automake bison file flex gcc gcc-c++ gettext-devel libtool libuuid-devel libxml2 libxml2-devel libxslt libxslt-devel lsof make openssl-devel pam-devel pax readline-devel tk-devel wget zlib-devel perl-Archive-Tar perl-XML-Parser libX11-devel libtool-ltdl-devel e2fsprogs-devel.x86_64 gcc-gfortran libicu-devel.x86_64 libgtextutils-devel.x86_64 perl-ExtUtils-AutoInstall.noarch perl-ExtUtils-Depends.noarch perl-ExtUtils-CBuilder.x86_64 perl-ExtUtils-CChecker.x86_64 perl-ExtUtils-Config.noarch perl-ExtUtils-DynaGlue.noarch perl-ExtUtils-Embed.x86_64 perl-ExtUtils-F77.noarch perl-ExtUtils-FakeConfig.noarch perl-ExtUtils-FindFunctions.noarch perl-ExtUtils-H2PM.noarch perl-ExtUtils-Helpers.noarch perl-ExtUtils-InstallPaths.noarch perl-ExtUtils-MakeMaker.x86_64 perl-ExtUtils-MakeMaker-Coverage.noarch perl-ExtUtils-ParseXS.x86_64 perl-ExtUtils-PerlPP.noarch perl-ExtUtils-PkgConfig.noarch perl-ExtUtils-TBone.noarch perl-ExtUtils-XSBuilder.noarch

Please make sure that the ntp package is installed: $ yum install ntp

User configuration

Add the esgf user:

$ adduser esgf
$ passwd esgf

configure esgf user with sudoers privileges. Add the following line to /etc/sudoers

esgf    ALL=(ALL)       ALL

Install the ESGF data/compute node

As reference guide the instructions used has been provided by IPSL1.

$ cd /usr/local/bin
$ wget -O esg-bootstrap
$ diff <(md5sum esg-bootstrap | tr -s " " | cut -d " " -f 1) <(curl -s | tr -s " " | cut -d " " -f 1) 
$ chmod 555 esg-bootstrap
$ esg-bootstrap --devel

Index peer configuration

Configure host certificate and CA public key

I think the procedure I sent in the below mail is correct but a little to technical. it might be better to use esg-node commands in order to federate:

Here is where you should put the signed csr we sent via scp to your server yesterday


Then if the tomcat key is not in /etc/grid-security, copy it inside:

$ cd /etc/grid-security; cp /esg/conf/tomcat/hostkey.pem ./

Then run this command to install the key pair in tomcat:

$ cd /etc/grid-security; esg-node --install-keypair hostkey.pem

You will be prompted to enter the cacert file; enter the url to the index node cacert.pem:

Please enter your Certificate Athority's certificate chain file(s): 
 [enter each cert file/url press return, press return with blank entry when done]

NoteBox(warn, This process should fetch the CA cert to `/etc/grid-security/certificates`, but doesn't. Yo need to manually copy the CA cert.?

Then set auto fetch certs false otherwise /etc/grid-security/certificates/* will be overwritten by esgf-prod peer groups certificates

$ esg-node --set-auto-fetch-certs false

Then restart your node

$ esg-node restart

Then register

$ esg-node --register

Then rebuild the Tomcat's trustore

$ esg-node --rebuild-truststore

Data Publishing

Attachments (8)

Download all attachments as: .zip