WRF for GRID (WRF4G) is a framework for the execution and monitoring of the WRF Modelling System in distributed computer resources (see this presentation). It provides a flexible and easy way of designing complex experiments involving many simulations (multiple start/end dates, multiparametric simulations, long climate runs, ...). The monitor allows a precise control of the state of the experiment, where broken simulations are automatically detected and relaunched on the next submission. WRF4G uses GridWay to access to heterogeneous resources in a transparent way.
Installation on the User Interface (UI)
General Requirements
WRF4G needs the following software preinstalled on the UI:
- GNU/Linux (x86_64)
- Python, version >= 2.4 and < 3.0
- GridFTP server
Download
Download and unpack the distribution file WRF4G.tar.gz to the installation directory, for example your $HOME directory.
[user@ui~]$ cd $HOME [user@ui~]$ wget http://meteo.macc.unican.es/work/WRF4G.tar.gz [user@ui~]$ tar xzvf WRF4G.tar.gz
Environment Configuration
Set up the environment variables "WRF4G_LOCATION" and "PATH" for WRF4G.
[user@ui~]$ export WRF4G_LOCATION=$WRF4G_INSTALL_PATH/WRF4G [user@ui~]$ export PATH=$WRF4G_LOCATION/bin:$PATH [user@ui~]$ export LD_LIBRARY_PATH=$WRF4G_LOCATION/lib/shared_libs:$LD_LIBRARY_PATH
And set up the enviroment for GISELA:
- Create a file (i.e. gisela_environment.sh) with these variables:
export LCG_CATALOG_TYPE=lfc export LFC_HOST=lfc.eela.ufrj.br export LCG_GFAL_INFOSYS=bdii.eela.ufrj.br:2170 export LFC_HOME=/grid/prod.vo.eu-eela.eu export VO="prod.vo.eu-eela.eu"
- Execute commands from gisela_environment.sh in the current shell environment.
[user@ui~]$ source gisela_environment.sh
GISELA Available Resources
lcg-infosites command can be used to obtain VO information on Grid resources. Before using lcg-infosites you have to execute gisela_environment.sh file for GISELA Infrastucture. Some using examples are showed below:
- Find out the CEs of your VO:
[user@ui~]$ lcg-infosites --vo $VO ce #CPU Free Total Jobs Running Waiting ComputingElement ---------------------------------------------------------- 16 16 0 0 0 gantt.cefet-rj.br:8443/cream-pbs-prod 21561 0 3409 7 3402 ce206.cern.ch:8443/cream-lsf-grid_eela 21561 826 3409 7 3402 ce204.cern.ch:8443/cream-lsf-grid_eela 21561 826 3409 7 3402 ce203.cern.ch:8443/cream-lsf-grid_eela 21561 0 3409 7 3402 ce205.cern.ch:8443/cream-lsf-grid_eela 21561 826 3409 7 3402 ce208.cern.ch:8443/cream-lsf-grid_eela 21561 0 3409 7 3402 ce207.cern.ch:8443/cream-lsf-grid_eela 21561 826 3409 7 3402 ce130.cern.ch:2119/jobmanager-lcglsf-grid_eela 21561 826 3409 7 3402 ce132.cern.ch:2119/jobmanager-lcglsf-grid_eela 21561 826 3409 7 3402 ce131.cern.ch:2119/jobmanager-lcglsf-grid_eela 21561 826 3409 7 3402 ce133.cern.ch:2119/jobmanager-lcglsf-grid_eela 260 107 4 4 0 ce01-tic.ciemat.es:2119/jobmanager-lcgpbs-prod_eela 1160 467 0 0 0 gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour 1160 467 0 0 0 gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym 1160 467 5 4 1 gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday 1160 467 2 2 0 gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday 10 10 0 0 0 ce01.unlp.edu.ar:2119/jobmanager-lcgpbs-long ...........
- Find out CE list with running jobs, free cpus, and maximum wallclock and CPU time.
[user@ui~]$ lcg-info --vo $VO --list-ce --attrs RunningJobs,FreeCPUs,MaxWCTime,MaxCPUTime - CE: axon-g01.ieeta.pt:2119/jobmanager-lcgpbs-prod - RunningJobs 0 - FreeCPUs 5 - MaxWCTime 4320 - MaxCPUTime 2880 - CE: cale.uniandes.edu.co:8443/cream-pbs-prod - RunningJobs 3 - FreeCPUs 94 - MaxWCTime 4320 - MaxCPUTime 2880 ...........
- Find out the SEs of your VO.
[user@ui~]$ lcg-infosites --vo $VO se Avail Space(Kb) Used Space(Kb) Type SEs ---------------------------------------------------------- 1258363960 8651392 n.a se.labmc.inf.utfsm.cl 288012854 11517683563 n.a lnx097.eela.if.ufrj.br 187037782 27605724 n.a se01.macc.unican.es
For more information you could execute lcg-infosites --help or see gLite information
Configuration of WRF4G to access to GISELA Resources
Next steps describe a specific configuration of the CEs for GISELA Infrastructure.
Firstly, you must remove default computing resource. Comment the line "mycomputer" (#) in $WRF4G_LOCATION/etc/framework4g.conf file.
GridWay configuration
If you want to configure Globus and CREAM resources of GISELA, you have to add the following lines to $WRF4G_LOCATION/opt/drm4g_gridway/etc/gwd.conf file:
# Example MAD Configuration for GISELA # GT2 IM_MAD = gisela_gt2:gw_im_mad_bdii:-q (GlueCEAccessControlBaseRule=VO\:prod.vo.eu-eela.eu)(GlueCEImplementationName=LCG-CE) -s bdii.eela.ufrj.br:tm_gt2:em_gt2 EM_MAD = em_gt2:gw_em_mad_gram2::rsl_nsh TM_MAD = tm_gt2:gw_tm_mad_dummy:-u gsiftp\://ui01.macc.unican.es # CREAM IM_MAD = gisela_cream:gw_im_mad_bdii:-q (GlueCEAccessControlBaseRule=VO\:prod.vo.eu-eela.eu)(GlueCEImplementationName=CREAM) -s bdii.eela.ufrj.br:tm_cream:em_cream EM_MAD = em_cream:gw_em_mad_cream.py::jdl TM_MAD = tm_cream:gw_tm_mad_dummy:-g
There are two options for the configuration of IM MAD:
- -q: it is possible to configure a GridWay instance to only use queues authorized to your VO by filtering them.
- -s: information server in a hierarchical configuration.
IM_MAD = glisela_gt2:gw_im_mad_bdii:-q (GlueCEAccessControlBaseRule=VO\:prod.vo.eu-eela.eu)(GlueCEImplementationName=LCG-CE) -s bdii.eela.ufrj.br:tm_gt2:em_gt2
There are two options for the configuration of the TM MAD:
- -g: starts a GASS server for each user.
TM_MAD = tm_gt2:gw_tm_mad_dummy:-g
- -u: specifies the URL of a GridFTP server running in the client. For example:
TM_MAD = tm_gt2:gw_tm_mad_dummy:-u gsiftp\://ui01.macc.unican.es
For more information about MAD configuration see GridWay Configuration Guide
Running environment configuration
The WRF4G_BASEPATH, WRF4G_DOMAINPATH, WRF4G_INPUT and WRF4G_APPS variables must be updated in $WRF4G_LOCATION/etc/resources.wrf4g (see resources.wrf4g for more details). The new supported URL syntaxes will be:
gsiftp://host[:port]/file
The example below shows an possible resources.wrf4g files:
WRF4G_VERSION="1.0beta" WRF_VERSION="3.1.1_r832INTEL_OMPI" WRF4G_BASEPATH="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/output" WRF4G_DOMAINPATH="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/domains" WRF4G_INPUT="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/input" WRF4G_APPS="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/apps" REQUIREMENTS='ARCH = "x86_64"' NP=1
Accessing the VOMS servers
To use the GISELA resources, the user should iniatilize the proxy through voms server:
[user@ui~]$ voms-proxy-init --voms prod.vo.eu-eela.eu Cannot find file or dir: /oceano/gmeteo/users/carlos/.glite/vomses Enter GRID pass phrase: Your identity: /DC=es/DC=irisgrid/O=unican/CN=josecarlos.blanco Creating temporary proxy ........................................ Done Contacting voms.eela.ufrj.br:15003 [/C=BR/O=ICPEDU/O=UFF BrGrid CA/O=UFRJ/OU=IF/CN=host/voms.eela.ufrj.br] "prod.vo.eu-eela.eu" Done Creating proxy ................................... Done Your proxy is valid until Tue Aug 23 22:15:06 2011
Testing installation
- Start up the WRF4G Framework. This command will start the services needed by WRF4G to run.
[user@ui~]$ wrf4g_framework start
- List the computing available resources.
[user@ui~]$ wrf4g_resources HID PRIO OS ARCH MHZ %CPU MEM(F/T) DISK(F/T) N(U/F/T) LRMS HOSTNAME 0 1 ScientificSLBor x86_6 3200 0 1024/1024 0/0 0/78/260 jobmanager-lcgpbs ce01-tic.ciemat.es 1 1 ScientificSLBer i686 1865 0 900/900 0/0 0/10/10 jobmanager-lcgpbs ce01.unlp.edu.ar 2 1 ScientificSLBer x86_6 1600 0 2048/2048 0/0 0/116/132 jobmanager-lcgpbs ce.labmc.inf.utfsm.cl 3 1 ScientificSLBer i686 2400 0 3072/3072 0/0 0/4/4 jobmanager-lcgpbs tochtli.nucleares.unam.mx 4 1 ScientificSLBer i686 2193 0 4096/4096 0/0 0/43/115 jobmanager-lcglsf grid012.ct.infn.it 5 1 Scientific Linu x86_6 2000 0 8150/8150 0/0 0/17/48 cream-pbs ce01.eela.if.ufrj.br 6 1 ScientificCERNS i386 2330 0 512/512 0/0 0/12/12 jobmanager-lcgpbs ce.cp.di.uminho.pt 7 1 CentOSFinal x86_6 2400 0 16000/16000 0/0 0/229/454 jobmanager-lcgpbs ce01.macc.unican.es 8 1 ScientificSLSL x86_6 2400 0 4058/4058 0/0 0/34/36 jobmanager-lcgsge ce01.up.pt 9 1 ScientificSLSL x86_6 2400 0 4058/4058 0/0 0/22/22 jobmanager-lcgsge grid001.fe.up.pt 10 1 ScientificSLBer i686 2330 0 2048/2048 0/0 0/18/18 cream-pbs gantt.cefet-rj.br 11 1 0 0 0/0 0/0 0/0/21818 cream-lsf ce206.cern.ch 12 1 0 0 0/0 0/0 0/833/21818 cream-lsf ce204.cern.ch 13 1 0 0 0/0 0/0 0/0/21818 cream-lsf ce205.cern.ch 14 1 0 0 0/0 0/0 0/0/21818 cream-lsf ce207.cern.ch 15 1 0 0 0/0 0/0 0/833/21818 cream-lsf ce208.cern.ch 16 1 CentOSFinal x86_6 2670 0 12000/12000 0/0 0/25/40 cream-pbs tochtli64.nucleares.unam.mx 17 1 Scientific Linu x86_6 2000 0 8178/8178 0/0 0/55/200 cream-pbs ce02.eela.if.ufrj.br 18 1 ScientificSLBor x86_6 3000 0 2048/2048 0/0 0/24/24 cream-pbs cream01.cecalc.ula.ve 19 1 0 0 0/0 0/0 0/0/0 ce.egee.di.uminho.pt 20 1 ScientificCERNS x86_6 1600 0 4096/4096 0/0 0/138/188 cream-pbs cale.uniandes.edu.co 21 1 ScientificSLSL x86_6 2400 0 4058/4058 0/0 0/20/22 cream-sge grid001.fc.up.pt
- Go to the folder where the test experiment is located:
[user@ui~]$ cd $WRF4G_LOCATION/experiments/single_test
- Prepare and submit the experiment:
[user@ui~]$ wrf4g_prepare [user@ui~]$ wrf4g_submit
- Run wrf4g_status to see the jobs' status
[user@ui~]$ wrf4g_status -e test Realization Status Chunks Comp.Res WN Run.Sta ext % testc Done 3/3 ce01.macc.unican.es wn002.macc.unican.es Finished 0 100.00
For more specific information about commands see WRF4G CLI
Problems
If you find any problem, please submit us a ticket!!
