wiki:WRF4GGiselaUI

Version 16 (modified by carlos, 10 years ago) (diff)

--

WRF for GRID (WRF4G) is a framework for the execution and monitoring of the WRF Modelling System in distributed computer resources (see this presentation). It provides a flexible and easy way of designing complex experiments involving many simulations (multiple start/end dates, multiparametric simulations, long climate runs, ...). The monitor allows a precise control of the state of the experiment, where broken simulations are automatically detected and relaunched on the next submission. WRF4G uses GridWay to access to heterogeneous resources in a transparent way.

Installation on the User Interface (UI)

General Requirements

WRF4G need the following software preinstalled on the UI:

  • GNU/Linux (x86_64)
  • Python, version >= 2.4 and < 3.0
  • MySQL for Python, MySQLdb
  • GridFTP server

Download

Download and unpack the distribution file WRF4G.tar.gz to the installation directory, for example your $HOME directory.

[user@ui~]$ cd $HOME
[user@ui~]$ wget http://meteo.macc.unican.es/work/WRF4G.tar.gz
[user@ui~]$ tar xzvf WRF4G.tar.gz

Environment Configuration

Set up the environment variables "WRF4G_LOCATION" and "PATH" for WRF4G.

[user@ui~]$ export WRF4G_LOCATION=$WRF4G_INSTALL_PATH/WRF4G
[user@ui~]$ export PATH=$WRF4G_LOCATION/bin:$PATH

And set up the enviroment for GISELA:

  • Create a file (i.e. gisela_environment.sh) with these variables:
    export LCG_CATALOG_TYPE=lfc
    export LFC_HOST=lfc.eela.ufrj.br
    export LCG_GFAL_INFOSYS=bdii.eela.ufrj.br:2170
    export LFC_HOME=/grid/prod.vo.eu-eela.eu
    export VO="prod.vo.eu-eela.eu"
    
  • Execute commands from gisela_environment.sh in the current shell environment.
    [user@ui~]$ source gisela_environment.sh
    

GISELA Available Resources

lcg-infosites command can be used to obtain VO information on Grid resources. Before using lcg-infosites you have to execute gisela_environment.sh file for GISELA Infrastucture. Some using examples are showed below:

  • Find out the CEs of your VO:
    [user@ui~]$ lcg-infosites --vo $VO ce 
    #CPU    Free    Total Jobs      Running Waiting ComputingElement
    ----------------------------------------------------------
      16      16       0              0        0    gantt.cefet-rj.br:8443/cream-pbs-prod
    21561      0    3409              7     3402    ce206.cern.ch:8443/cream-lsf-grid_eela
    21561    826    3409              7     3402    ce204.cern.ch:8443/cream-lsf-grid_eela
    21561    826    3409              7     3402    ce203.cern.ch:8443/cream-lsf-grid_eela
    21561      0    3409              7     3402    ce205.cern.ch:8443/cream-lsf-grid_eela
    21561    826    3409              7     3402    ce208.cern.ch:8443/cream-lsf-grid_eela
    21561      0    3409              7     3402    ce207.cern.ch:8443/cream-lsf-grid_eela
    21561    826    3409              7     3402    ce130.cern.ch:2119/jobmanager-lcglsf-grid_eela
    21561    826    3409              7     3402    ce132.cern.ch:2119/jobmanager-lcglsf-grid_eela
    21561    826    3409              7     3402    ce131.cern.ch:2119/jobmanager-lcglsf-grid_eela
    21561    826    3409              7     3402    ce133.cern.ch:2119/jobmanager-lcglsf-grid_eela
     260     107       4              4        0    ce01-tic.ciemat.es:2119/jobmanager-lcgpbs-prod_eela
    1160     467       0              0        0    gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour
    1160     467       0              0        0    gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym
    1160     467       5              4        1    gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday
    1160     467       2              2        0    gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday
      10      10       0              0        0    ce01.unlp.edu.ar:2119/jobmanager-lcgpbs-long
    ...........
    
  • Find out CE list with running jobs, free cpus, and maximum wallclock and CPU time.
    [user@ui~]$ lcg-info --vo $VO --list-ce --attrs RunningJobs,FreeCPUs,MaxWCTime,MaxCPUTime
    - CE: axon-g01.ieeta.pt:2119/jobmanager-lcgpbs-prod
      - RunningJobs         0
      - FreeCPUs            5
      - MaxWCTime           4320
      - MaxCPUTime          2880
    
    - CE: cale.uniandes.edu.co:8443/cream-pbs-prod
      - RunningJobs         3
      - FreeCPUs            94
      - MaxWCTime           4320
      - MaxCPUTime          2880
    ...........
    
  • Find out the SEs of your VO.
    [user@ui~]$ lcg-infosites --vo $VO se
    Avail Space(Kb) Used Space(Kb)  Type    SEs
    ----------------------------------------------------------
    1258363960      8651392         n.a     se.labmc.inf.utfsm.cl
    288012854       11517683563     n.a     lnx097.eela.if.ufrj.br
    187037782       27605724        n.a     se01.macc.unican.es
    

For more information you could execute lcg-infosites --help or see gLite information

Configuration of WRF4G to access to GISELA Resources

Next steps describe a specific configuration of the CEs for GISELA Infrastructure.

Firsly

GridWay configuration

If you want to configure Globus and CREAM resources of GISELA, you have to add the following lines to "$WRF4G_LOCATION/opt/drm4g_gridway/etc/gwd.conf" file:

# Example MAD Configuration for GISELA
# GT2 
IM_MAD = gisela_gt2:gw_im_mad_bdii:-q (GlueCEAccessControlBaseRule=VO\:prod.vo.eu-eela.eu)(GlueCEImplementationName=LCG-CE)  -s bdii.eela.ufrj.br:tm_gt2:em_gt2
EM_MAD = em_gt2:gw_em_mad_gram2::rsl_nsh
TM_MAD = tm_gt2:gw_tm_mad_dummy:-u gsiftp\://ui01.macc.unican.es

# CREAM
IM_MAD = gisela_cream:gw_im_mad_bdii:-q (GlueCEAccessControlBaseRule=VO\:prod.vo.eu-eela.eu)(GlueCEImplementationName=CREAM)  -s bdii.eela.ufrj.br:tm_cream:em_cream
EM_MAD = em_cream:gw_em_mad_cream.py::jdl
TM_MAD = tm_cream:gw_tm_mad_dummy:-g

There are two options for the configuration of IM MAD:

  • -q: it is possible to configure a GridWay instance to only use queues authorized to your VO by filtering them.
  • -s: information server in a hierarchical configuration.
    IM_MAD = glisela_gt2:gw_im_mad_bdii:-q (GlueCEAccessControlBaseRule=VO\:prod.vo.eu-eela.eu)(GlueCEImplementationName=LCG-CE) -s bdii.eela.ufrj.br:tm_gt2:em_gt2
    

There are two options for the configuration of the TM MAD:

  • -g: starts a GASS server for each user.
    TM_MAD = tm_gt2:gw_tm_mad_dummy:-g
    
  • -u: specifies the URL of a GridFTP server running in the client. For example:
    TM_MAD = tm_gt2:gw_tm_mad_dummy:-u gsiftp\://ui01.macc.unican.es
    

For more information about MAD configuration see GridWay Configuration Guide

Running environment configuration

The "WRF4G_BASEPATH", "WRF4G_DOMAINPATH", "WRF4G_INPUT" and "WRF4G_APPS" variables must be updated in resources.wrf4g under the folder $WRF4G_LOCATION/etc. The new supported URL syntaxes will be:

gsiftp://host[:port]/file

The example below shows an possible resources.wrf4g files:

WRF4G_VERSION="1.0beta"
WRF_VERSION="3.1.1_r832INTEL_OMPI"
WRF4G_BASEPATH="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/output" 
WRF4G_DOMAINPATH="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/domains"
WRF4G_INPUT="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/input"
WRF4G_APPS="gsiftp://ui01.macc.unican.es/$WRF4G_LOCATION/repository/apps"
NP=1

Accessing the VOMS servers

To use the GISELA resources, the user should iniatilize the proxy through voms server:

[user@ui~]$ voms-proxy-init --voms prod.vo.eu-eela.eu
Cannot find file or dir: /oceano/gmeteo/users/carlos/.glite/vomses
Enter GRID pass phrase:
Your identity: /DC=es/DC=irisgrid/O=unican/CN=josecarlos.blanco
Creating temporary proxy ........................................ Done
Contacting  voms.eela.ufrj.br:15003 [/C=BR/O=ICPEDU/O=UFF BrGrid CA/O=UFRJ/OU=IF/CN=host/voms.eela.ufrj.br] "prod.vo.eu-eela.eu" Done
Creating proxy ................................... Done
Your proxy is valid until Tue Aug 23 22:15:06 2011

Testing installation

  • Start up the WRF4G Framework. This command will start the services needed by WRF4G to run.
    [user@ui~]$ wrf4g_framework start
    
  • List the computing available resources.
    [user@ui~]$ wrf4g_resources 
    HID PRIO OS              ARCH  MHZ  %CPU    MEM(F/T)     DISK(F/T)      N(U/F/T) LRMS               HOSTNAME                                             
    0   1    ScientificSLBor x86_6 3200    0   1024/1024           0/0      0/78/260 jobmanager-lcgpbs  ce01-tic.ciemat.es            
    1   1    ScientificSLBer i686  1865    0     900/900           0/0       0/10/10 jobmanager-lcgpbs  ce01.unlp.edu.ar              
    2   1    ScientificSLBer x86_6 1600    0   2048/2048           0/0     0/116/132 jobmanager-lcgpbs  ce.labmc.inf.utfsm.cl         
    3   1    ScientificSLBer i686  2400    0   3072/3072           0/0         0/4/4 jobmanager-lcgpbs  tochtli.nucleares.unam.mx     
    4   1    ScientificSLBer i686  2193    0   4096/4096           0/0      0/43/115 jobmanager-lcglsf  grid012.ct.infn.it            
    5   1    Scientific Linu x86_6 2000    0   8150/8150           0/0       0/17/48 cream-pbs          ce01.eela.if.ufrj.br          
    6   1    ScientificCERNS i386  2330    0     512/512           0/0       0/12/12 jobmanager-lcgpbs  ce.cp.di.uminho.pt            
    7   1    CentOSFinal     x86_6 2400    0 16000/16000           0/0     0/229/454 jobmanager-lcgpbs  ce01.macc.unican.es           
    8   1    ScientificSLSL  x86_6 2400    0   4058/4058           0/0       0/34/36 jobmanager-lcgsge  ce01.up.pt                    
    9   1    ScientificSLSL  x86_6 2400    0   4058/4058           0/0       0/22/22 jobmanager-lcgsge  grid001.fe.up.pt              
    10  1    ScientificSLBer i686  2330    0   2048/2048           0/0       0/18/18 cream-pbs          gantt.cefet-rj.br             
    11  1                             0    0         0/0           0/0     0/0/21818 cream-lsf          ce206.cern.ch                 
    12  1                             0    0         0/0           0/0   0/833/21818 cream-lsf          ce204.cern.ch                 
    13  1                             0    0         0/0           0/0     0/0/21818 cream-lsf          ce205.cern.ch                 
    14  1                             0    0         0/0           0/0     0/0/21818 cream-lsf          ce207.cern.ch                 
    15  1                             0    0         0/0           0/0   0/833/21818 cream-lsf          ce208.cern.ch                 
    16  1    CentOSFinal     x86_6 2670    0 12000/12000           0/0       0/25/40 cream-pbs          tochtli64.nucleares.unam.mx   
    17  1    Scientific Linu x86_6 2000    0   8178/8178           0/0      0/55/200 cream-pbs          ce02.eela.if.ufrj.br          
    18  1    ScientificSLBor x86_6 3000    0   2048/2048           0/0       0/24/24 cream-pbs          cream01.cecalc.ula.ve         
    19  1                             0    0         0/0           0/0         0/0/0                    ce.egee.di.uminho.pt          
    20  1    ScientificCERNS x86_6 1600    0   4096/4096           0/0     0/138/188 cream-pbs          cale.uniandes.edu.co          
    21  1    ScientificSLSL  x86_6 2400    0   4058/4058           0/0       0/20/22 cream-sge          grid001.fc.up.pt  
    

  • Go to the folder where the test experiment is located:
    [user@ui~]$ cd $WRF4G_LOCATION/experiments/single_test
    
  • Prepare and submit the experiment:
    [user@ui~]$ wrf4g_prepare
    [user@ui~]$ wrf4g_submit
    
  • Run wrf4g_status to see the jobs' status
    [user@ui~]$ wrf4g_status -e test 
    Realization Status   Chunks     Comp.Res            WN                    Run.Sta       ext   %
    testc       Done     3/3        ce01.macc.unican.es wn002.macc.unican.es  Finished      0     100.00
    

For more specific information about commands see WRF4G CLI

Problems

If you find any problem, please submit us a ticket!!