Changes between Version 43 and Version 44 of ESGFGetCredentials

Timestamp:

Jan 8, 2015 12:46:26 AM (7 years ago)

Author:

antonio

Comment:

--

ESGFGetCredentials

@@ -105 +105 @@
 
 ----
-= Specific use cases =
-
+= Use cases =
+
+Some enviroment varibles can be set: 
+1. The path where the user's credentials and ESGF peers certificates will be retrived
+{{{
+ESGF_HOME=.esg
+}}}
+1. The user's !OpenId and password
+{{{
+OPENID=https://esgf-data.dkrz.de/esgf-idp/openid/testuser
+OPENID_PASS=userpassword
+}}}
+
+For convenience the user's credentials and trust certificates will be retrived in JKS and PEM formats:
+{{{
+java -jar getESGFCredentials.jar --openid $OPENID --password $OPENID_PASS --writeall --output $ESGF_HOME
+}}}
+
+In the following use cases this URLs will be used:
+1. A URL for HTTP file dowloading
+{{{
+HTTP_URL=http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
+}}}
+2. A URL for DODS/OPeNDAP access
+{{{
+DODS_URL=http://esgf-data1.ceda.ac.uk/thredds/dodsC/esg_dataroot/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ua/ua_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2066010103-2075123121.nc
+}}}
 == Aria2 ==
 
+[http://aria2.sourceforge.net aria2] is a lightweight multi-protocol & multi-source command-line download utility. It supports HTTP/HTTPS, FTP, BitTorrent and Metalink. aria2 can be manipulated via built-in JSON-RPC and XML-RPC interfaces.
+=== File download ===
+
+{{{
+aria2c --private-key=$ESGF_HOME/credentials.pem --certificate=$ESGF_HOME/credentials.pem --check-certificate=true --ca-certificate=$ESGF_HOME/ca-certificates.pem $HTTP_URL
+}}}
+
+=== Retriving files from Metalink ===
 1. Get a metalink of ESGF Files
  * Download this metalink file -> [raw-attachment:ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1.metalink example_metalink][[BR]]
@@ -125 +158 @@
 == cURL ==
 
-1. Retrieve ESGF credentials in current directory 
-{{{
-java -jar getESGFCredentials-0.1.jar --openid <your_openid> --password <your_password> --credentials --cacertspem --output .
-}}}
-
-2. Use curl to download the file[[BR]]
-   example_file: http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
-{{{
-curl -L -C - --cookie-jar curl-cookie --cookie curl-cookie --cert credentials.pem --cacert ca-certificates.pem -O http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
+=== File download ===
+{{{
+curl --location --continue-at - --cookie curl-cookie --cert $ESGF_HOME/credentials.pem --cacert $ESGF_HOME/ca-certificates.pem -O $HTTP_URL
 }}}
 
@@ -145 +172 @@
      * '''-O''' (`-O/--remote-name`) to write output to a local file named like the remote file we get. You can use (`-o/--output` <file-name> option) to specify the name of the file.
     
-== WGET (OPENSSL) ==
-1. Retrieve ESGF credentials in $HOME/.esg
-{{{
-java -jar getESGFCredentials-0.1.jar --openid <openid> --password <password> --credentials --cacerts
-}}}
-
-2. Wget file[[BR]]
-   example_file: http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
-{{{
-wget -c --ca-directory=$HOME/.esg/certificates --certificate=$HOME/.esg/credentials.pem --private-key=$HOME/.esg/credentials.pem --save-cookies=$HOME/.esg/cookies --load-cookies=$HOME/.esg/cookies ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
-}}}
-
-
-== WGET (GNU TLS) ==
-
-1. Retrieve ESGF credentials in $HOME/.esg
-{{{
-java -jar getESGFCredentials-0.1.jar --openid <openid> --password <password> --credentials --cacertspem
-}}}
-
-2. Wget file[[BR]]
-   example_file: http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
-{{{
-wget -c --certificate=$HOME/.esg/credentials.pem --private-key=$HOME/.esg/credentials.pem --save-cookies=$HOME/.esg/cookies --load-cookies=$HOME/.esg/cookies --ca-certificate=$HOME/.esg/ca-certificates.pem ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc
-}}}
-
-== ToolsUI ==
-
-1. Retrieve ESGF credentials in $HOME/.esg
-{{{
-java -jar getESGFCredentials-0.1.jar --openid <openid> --password <password> --keystorejks --cacertsjks
-}}}
-
-2. Start ToolsUI
-{{{
-java -Dkeystore=$HOME/keystore_jks.ks -Dkeystorepassword=changeit -Dtruststore=$HOME/.esg/esg-truststore.ts -Dtruststorepassword=changeit -jar toolsUI-4.3.jar
-}}}
-
-3. This link is a ESGF OPENDAP resource http://data.meteo.unican.es/thredds/dodsC/esg_cordexnoncommercial/cordex/output/EUR-22/UCAN/ECMWF-ERAINT/evaluation/r1i1p1/UCAN-WRF331G/v02/3hr/clt/v20140616/clt_EUR-22_ECMWF-ERAINT_evaluation_r1i1p1_UCAN-WRF331G_v02_3hr_19790101-19791231.nc
-
-For more: http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/Session.html
+== GNU Wget ==
+
+== NetCDF-C ==
+
+The NetCDF-C libray from version 4.1 can be compiled with DAP support. Check with `nc-config` command if your NetCDF libray has been compiled with DAP support. See [https://www.unidata.ucar.edu/software/netcdf/docs/netcdf/DAP-Support.html].
+
+DAP access is based on [http://curl.haxx.se/libcurl/ libcurl] library. The configuration paramenters are based on a file named `.dodsrc` existing in the current working directory or user's home
+
+{{{
+echo -e ' HTTP.SSL.VALIDATE=1 \n HTTP.SSL.CAPATH=$ESGF_HOME/certificates \n HTTP.SSL.CERTIFICATE=$ESGF_HOME/credentials.pem' > .dodsrc && ncdump -h $DODS_URL
+}}}
+
+== NetCDF-Java ==
+
+The NetCDF-Java library can use the credentials and trust store by defining JVM properties as command line arguments:
+{{{
+NCJ_PROP=-Dkeystore=$ESGF_HOME/keystore_jks.ks -Dkeystorepassword=changeit -Dtruststore=$ESGF_HOME/esg-truststore.ts -Dtruststorepassword=changeit
+}}}
+
+For more info visit [http://www.unidata.ucar.edu/software/thredds/current/netcdf-java]
+
+=== NCdumpW ===
+
+Dump DODS/OPeNDAP URL metadata:
+
+{{{
+java $NCJ_PROP -cp netcdf-java/toolsUI-4.3.jar ucar.nc2.NCdumpW $DODS_URL -cdl
+}}}
+
+=== ToolsUI ===
+
+Open a dataset (i.e. DODS/OPeNDAP) with NetCDF-Java's ToolsUI Java application:
+{{{
+java $NCJ -jar netcdf-java/toolsUI-4.3.jar $DODS_URL
+}}}
 
 == ESGF WGET Script (Linux) ==
@@ -197 +218 @@
 }}}
 
-== NETCDF DAP ==
-https://www.unidata.ucar.edu/software/netcdf/docs/netcdf/DAP-Support.html
-{{{
-java -jar getESGFCredentials-0.0.2.jar -o <openid> -p <password> --credentials --cacerts --cacertspem --output <path>
-}}}
 = Developers Guide =
 == Github ==