107 | | = Specific use cases = |
108 | | |
| 107 | = Use cases = |
| 108 | |
| 109 | Some enviroment varibles can be set: |
| 110 | 1. The path where the user's credentials and ESGF peers certificates will be retrived |
| 111 | {{{ |
| 112 | ESGF_HOME=.esg |
| 113 | }}} |
| 114 | 1. The user's !OpenId and password |
| 115 | {{{ |
| 116 | OPENID=https://esgf-data.dkrz.de/esgf-idp/openid/testuser |
| 117 | OPENID_PASS=userpassword |
| 118 | }}} |
| 119 | |
| 120 | For convenience the user's credentials and trust certificates will be retrived in JKS and PEM formats: |
| 121 | {{{ |
| 122 | java -jar getESGFCredentials.jar --openid $OPENID --password $OPENID_PASS --writeall --output $ESGF_HOME |
| 123 | }}} |
| 124 | |
| 125 | In the following use cases this URLs will be used: |
| 126 | 1. A URL for HTTP file dowloading |
| 127 | {{{ |
| 128 | HTTP_URL=http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
| 129 | }}} |
| 130 | 2. A URL for DODS/OPeNDAP access |
| 131 | {{{ |
| 132 | DODS_URL=http://esgf-data1.ceda.ac.uk/thredds/dodsC/esg_dataroot/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ua/ua_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2066010103-2075123121.nc |
| 133 | }}} |
127 | | 1. Retrieve ESGF credentials in current directory |
128 | | {{{ |
129 | | java -jar getESGFCredentials-0.1.jar --openid <your_openid> --password <your_password> --credentials --cacertspem --output . |
130 | | }}} |
131 | | |
132 | | 2. Use curl to download the file[[BR]] |
133 | | example_file: http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
134 | | {{{ |
135 | | curl -L -C - --cookie-jar curl-cookie --cookie curl-cookie --cert credentials.pem --cacert ca-certificates.pem -O http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
| 160 | === File download === |
| 161 | {{{ |
| 162 | curl --location --continue-at - --cookie curl-cookie --cert $ESGF_HOME/credentials.pem --cacert $ESGF_HOME/ca-certificates.pem -O $HTTP_URL |
147 | | == WGET (OPENSSL) == |
148 | | 1. Retrieve ESGF credentials in $HOME/.esg |
149 | | {{{ |
150 | | java -jar getESGFCredentials-0.1.jar --openid <openid> --password <password> --credentials --cacerts |
151 | | }}} |
152 | | |
153 | | 2. Wget file[[BR]] |
154 | | example_file: http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
155 | | {{{ |
156 | | wget -c --ca-directory=$HOME/.esg/certificates --certificate=$HOME/.esg/credentials.pem --private-key=$HOME/.esg/credentials.pem --save-cookies=$HOME/.esg/cookies --load-cookies=$HOME/.esg/cookies ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
157 | | }}} |
158 | | |
159 | | |
160 | | == WGET (GNU TLS) == |
161 | | |
162 | | 1. Retrieve ESGF credentials in $HOME/.esg |
163 | | {{{ |
164 | | java -jar getESGFCredentials-0.1.jar --openid <openid> --password <password> --credentials --cacertspem |
165 | | }}} |
166 | | |
167 | | 2. Wget file[[BR]] |
168 | | example_file: http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
169 | | {{{ |
170 | | wget -c --certificate=$HOME/.esg/credentials.pem --private-key=$HOME/.esg/credentials.pem --save-cookies=$HOME/.esg/cookies --load-cookies=$HOME/.esg/cookies --ca-certificate=$HOME/.esg/ca-certificates.pem ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc http://wdcc-esgf.dkrz.de:8080/ESGF/fileServer/cmip5/output1/IPSL/IPSL-CM5A-LR/esmrcp85/6hr/atmos/6hrPlev/r1i1p1/v20120114/ta/ta_6hrPlev_IPSL-CM5A-LR_esmrcp85_r1i1p1_2016010103-2025123121.nc |
171 | | }}} |
172 | | |
173 | | == ToolsUI == |
174 | | |
175 | | 1. Retrieve ESGF credentials in $HOME/.esg |
176 | | {{{ |
177 | | java -jar getESGFCredentials-0.1.jar --openid <openid> --password <password> --keystorejks --cacertsjks |
178 | | }}} |
179 | | |
180 | | 2. Start ToolsUI |
181 | | {{{ |
182 | | java -Dkeystore=$HOME/keystore_jks.ks -Dkeystorepassword=changeit -Dtruststore=$HOME/.esg/esg-truststore.ts -Dtruststorepassword=changeit -jar toolsUI-4.3.jar |
183 | | }}} |
184 | | |
185 | | 3. This link is a ESGF OPENDAP resource http://data.meteo.unican.es/thredds/dodsC/esg_cordexnoncommercial/cordex/output/EUR-22/UCAN/ECMWF-ERAINT/evaluation/r1i1p1/UCAN-WRF331G/v02/3hr/clt/v20140616/clt_EUR-22_ECMWF-ERAINT_evaluation_r1i1p1_UCAN-WRF331G_v02_3hr_19790101-19791231.nc |
186 | | |
187 | | For more: http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/reference/Session.html |
| 174 | == GNU Wget == |
| 175 | |
| 176 | == NetCDF-C == |
| 177 | |
| 178 | The NetCDF-C libray from version 4.1 can be compiled with DAP support. Check with `nc-config` command if your NetCDF libray has been compiled with DAP support. See [https://www.unidata.ucar.edu/software/netcdf/docs/netcdf/DAP-Support.html]. |
| 179 | |
| 180 | DAP access is based on [http://curl.haxx.se/libcurl/ libcurl] library. The configuration paramenters are based on a file named `.dodsrc` existing in the current working directory or user's home |
| 181 | |
| 182 | {{{ |
| 183 | echo -e ' HTTP.SSL.VALIDATE=1 \n HTTP.SSL.CAPATH=$ESGF_HOME/certificates \n HTTP.SSL.CERTIFICATE=$ESGF_HOME/credentials.pem' > .dodsrc && ncdump -h $DODS_URL |
| 184 | }}} |
| 185 | |
| 186 | == NetCDF-Java == |
| 187 | |
| 188 | The NetCDF-Java library can use the credentials and trust store by defining JVM properties as command line arguments: |
| 189 | {{{ |
| 190 | NCJ_PROP=-Dkeystore=$ESGF_HOME/keystore_jks.ks -Dkeystorepassword=changeit -Dtruststore=$ESGF_HOME/esg-truststore.ts -Dtruststorepassword=changeit |
| 191 | }}} |
| 192 | |
| 193 | For more info visit [http://www.unidata.ucar.edu/software/thredds/current/netcdf-java] |
| 194 | |
| 195 | === NCdumpW === |
| 196 | |
| 197 | Dump DODS/OPeNDAP URL metadata: |
| 198 | |
| 199 | {{{ |
| 200 | java $NCJ_PROP -cp netcdf-java/toolsUI-4.3.jar ucar.nc2.NCdumpW $DODS_URL -cdl |
| 201 | }}} |
| 202 | |
| 203 | === ToolsUI === |
| 204 | |
| 205 | Open a dataset (i.e. DODS/OPeNDAP) with NetCDF-Java's ToolsUI Java application: |
| 206 | {{{ |
| 207 | java $NCJ -jar netcdf-java/toolsUI-4.3.jar $DODS_URL |
| 208 | }}} |