wiki:WRF4G2.0/CLI

CLI

wrf4g is a WRF4G's Command Line Interface (CLI) for managing WRF experiments.

Available commands

Type --help option obtain information about all commands:

$ wrf4g --help
WRF4G is a framework for the execution and monitoring of the WRF Modelling System 
on Distributed Computing Infrastructure (DCIs onwards). For additional information, see http://meteo.unican.es/trac/wiki/WRF4G

Usage: wrf4g [ --version ] [ -h | --help ] [ --dbg ]
             <command> [ <args>... ]

Options:
    -h --help  Show help.
    --version  Show version.
    --dbg      Debug mode.
    
wrf4g commands are:
   start       Start DRM4G daemon and ssh-agent.
   stop        Stop  DRM4G daemon and ssh-agent.
   status      Check DRM4G daemon and ssh-agent.
   syncdb      Create WRF4G database tables on the configured database.  
   conf        Configure DRM4G daemon and scheduler, database and logger parameters.
   exp         Manage WRF4G experiments.
   rea         Manage WRF4G realizations.
   resource    Manage computing resources.
   id          Manage identities for resources.
   host        Print information about the DCI hosts.
   job         Get status and history and cancel jobs.
 
See 'wrf4g <command> --help' for more information on a specific command.

start :

Start DRM4G daemon and ssh-agent. 
    
Usage: 
    wrf4g start [ --dbg ] [ --clear-conf ] [ --disc-jobs ] 
            
Options:
   --dbg                Debug mode.
   --clear-conf         Clear WRF4G's settings stored in .wrf4g directory.
   --disc-jobs          All available jobs on WRF4G will be discarded. 

stop:

Stop DRM4G daemon and ssh-agent. 
    
Usage: 
    wrf4g stop [ --dbg ] 
   
Options:
   --dbg    Debug mode.

status:

Check DRM4G daemon and ssh-agent. 
    
Usage: 
    wrf4g status [ --dbg ] 
   
Options:
   --dbg    Debug mode.

conf:

Configure DRM4G daemon, scheduler, database and logging parameters.
                
Usage:  
   wrf4g conf ( daemon | sched | logger | database ) [ --dbg ]

Options:
   --dbg    Debug mode

exp :

Manage WRF4G experiments. 
    
Usage: 
    wrf4g exp list                [ --pattern=<name> ]
    wrf4g exp <name> define       [ --dbg ] [ --force ]   [ --from-template=<name> ] [ --dir=<directory> ] 
    wrf4g exp <name> edit         [ --dbg ] 
    wrf4g exp <name> create       [ --dbg ] [ --dry-run ] [ --dir=<directory> ]
    wrf4g exp <name> update       [ --dbg ] [ --dry-run ] 
    wrf4g exp <name> submit       [ --dbg ] [ --dry-run ] [ --priority=<value> ] [ --pattern=<name> ] [ --rea-state=<state> ] [ --rerun ] 
    wrf4g exp <name> status       [ --dbg ] [ --pattern=<name> ] [ --rea-state=<state> ] [ --delay=<seconds> ]  
    wrf4g exp <name> statistics   [ --pattern=<name> ]
    wrf4g exp <name> cancel       [ --dbg ] [ --dry-run ] [ --pattern=<name> ] [ --rea-state=<state> ] [ --hard ]
    wrf4g exp <name> set-priority [ --dbg ] [ --dry-run ] [ --pattern=<name> ] <priority>
    wrf4g exp <name> delete       [ --dbg ] [ --dry-run ] 
   
Options:
    --dbg                     Debug mode.
    -n --dry-run              Dry run.
    -f --force                Force to remove if it exists.
    -P --priority=<value>     Fix-priority for scheduling [default: 0]. 
    -p --pattern=<name>       Pattern to find experiments and realizations.
    -s --rea-state=<state>    Select only realizations in the indicated state. Available states :
                              PREPARED, SUBMITTED, RUNNING, PENDING, FAILED and FINISHED 
    -t --from-template=<name> Experiment template, avaible templates are default, single, physics. 
    -d --dir=<directory>      Directory to create or start an experiment [default: ./].
    --delay=<seconds>         Refresh experiment information every delay seconds.
    --rerun                   Force to run although this realization or experiment has finished.
    --hard                    Remove jobs from without synchronizing.
  
Commands:
    list                      Show all the experiments available.
    define                    Create the files needed to define a WRF4G experiment.
    edit                      Edit experiment.wrf4g file.
    create                    Given experiment.wrf4g file, prepare the experiment creating 
                              the realizations and chunks needed.
    update                    Update the experiment configuration.
    submit                    Submit the experiment.
    status                    Check the status of realizations and chunks showing computing resources, 
                              job identifier and exit codes (SEE EXIT CODES) 
    cancel                    Cancel the active realizations by killing their jobs.
    set-priority              Change the scheduling priority of any job releted to a realization. 
                              The priority must be in range [0,20], and the default value is 0. 
                              When a job gets a priority of 20, it becomes an urgent job. This job 
                              is dispatched as soon as possible passing all the scheduling policies.
    delete                    Remove the experiment from the database.

EXIT CODES
    1  : Error creating log directory        
    2  : Error copying apps            
    3  : Error app type does not exist            
    4  : Error executing source script       
    5  : Job already executed 
    6  : Error creating directory to simulate
    7  : Error finding WRF binaries 
    8  : Error copying restart files        
    9  : There is a mismatch in the restart date   
    10 : Error copying files    
    11 : Error downloading WPS files    
    12 : Error copying boundaries           
    13 : Error modifying namelist
    14 : Error executing PREPROCESSOR
    15 : Error linking GRIB files     
    16 : Error executing UNGRIB
    17 : Error executing METGRID       
    18 : Error executing REAL
    19 : Error uploading WPS files      
    20 : Error executing WRF
    21 : Error executing POSTPROCESSOR 
    22 : Error copying output file    
    23 : Job killed by the system 
    255: Unexpected error

rea:

Manage WRF4G realizations. 
    
Usage: 
     wrf4g rea <name> submit       [ --dbg ] [ --dry-run ] [ --priority=<value> ] [ --rerun ] [ <first_ch> [ <last_ch> ] ]
     wrf4g rea <name> status       [ --dbg ] [ --delay=<seconds> ]
     wrf4g rea <name> info
     wrf4g rea <name> log          [ --dbg ] [ --dir=<directory> ] <chunk_id>
     wrf4g rea <name> set-priority [ --dbg ] [ --dry-run ] <priority>
     wrf4g rea <name> cancel       [ --dbg ] [ --dry-run ] [ --hard ]
     wrf4g rea <name> set-restart  [ --dbg ] <date>
     wrf4g rea <name> get-restart  
   
Options:
    --dbg                  Debug mode.
    -n --dry-run           Dry run.
    --rerun                Force to run although the realization has finished.
    -P --priority=<value>  Fix-priority for scheduling [default: 0].
    --delay=<seconds>      Refresh experiment information every delay seconds.    
    -d --dir=<directory>   Directory to unpack log files [default: ./].
    --hard                 Remove jobs from without synchronizing.
  
Commands:
    submit                 Submit the realization.       
    status                 Check the status of a realization showing computing resources, 
                           job identifier and exit codes (SEE EXIT CODES).
    info                   Get information about configuration.
    log                    Get log files from a chunk.
    set-priority           Change the scheduling priority of any job releted to the realization. 
                           The priority must be in range [0,20], and the default value is 0. 
                           When a job gets a priority of 20, it becomes an urgent job, and it is 
                           dispatched as soon as possible passing all the scheduling policies. 
    cancel                 Cancel the realization by killing its jobs.
    set-restart            Modify realization restart date.
    get-restart            Show realization restart date.

EXIT CODES
    1  : Error creating log directory        
    2  : Error copying apps            
    3  : Error app type does not exist            
    4  : Error executing source script       
    5  : Job already executed 
    6  : Error creating directory to simulate
    7  : Error finding WRF binaries 
    8  : Error copying restart files        
    9  : There is a mismatch in the restart date   
    10 : Error copying files    
    11 : Error downloading WPS files    
    12 : Error copying boundaries           
    13 : Error modifying namelist
    14 : Error executing PREPROCESSOR
    15 : Error linking GRIB files     
    16 : Error executing UNGRIB
    17 : Error executing METGRID       
    18 : Error executing REAL
    19 : Error uploading WPS files      
    20 : Error executing WRF
    21 : Error executing POSTPROCESSOR 
    22 : Error copying output file    
    23 : Job killed by the system 
    255: Unexpected error

job :

Get status and history and cancel jobs.

Usage: 
    wrf4g job list    [ --dbg ] [ --delay=<seconds> ] [ <job_id> ] 
    wrf4g job cancel  [ --dbg ] [ --hard ] <job_id>  
    wrf4g job log     [ --dbg ] <job_id>
    wrf4g job history [ --dbg ] <job_id> 
   
Arguments:
   <job_id>               Job identifier.

Options:
   --dbg                  Debug mode.
   --delay=<seconds>      Refresh experiment information every delay seconds.
   --hard                 Remove jobs from without synchronizing.
    
Commands:
   list                   Monitor jobs previously submitted.
   cancel                 Cancel jobs.
   log                    Keep track of a job.
   history                Get information about the execution history of a job.

Job field information:
   JID                    Job identification.
   DM                     Dispatch Manager state, one of: 
                                pend, hold, prol, prew, wrap, epil, canl, stop, migr, done, fail.
   EM                     Execution Manager state: pend, susp, actv, fail, done.
   START                  The time the job entered the system.
   END                    The time the job reached a final state (fail or done).
   EXEC                   Total execution time, includes suspension time in the remote queue system.
   XFER                   Total file transfer time, includes stage-in and stage-out phases.
   EXIT                   Job exit code.
   TEMPLATE               Filename of the job template used for this job.
   HOST                   Hostname where the job is being executed.
   HID                    Host identification.
   PROLOG                 Total prolog (file stage-in phase) time.
   WRAPPER                Total wrapper (execution phase) time.
   EPILOG                 Total epilog (file stage-out esphase) time.
   MIGR                   Total migration time.
   REASON                 The reason why the job left this host.
   QUEUE                  Queue name.

vcp:

Virtual copy a command to copy files using different protocols.
        
Usage:
    wrf4g vcp [ --dbg ] [ --overwrite  ] <source> <dest>

Arguments:

Options:
    --dbg           Debug mode.
    -o --overwrite  If the destination already exists, it will be overwritten.
    
Supported protocols:        
    LFN             lfn:///grid/VO/file file://home/user/file 
    GRIDFTP         gridftp://computer:2812/grid/VO/user/file
    RSYNC           rsync://user@computer:34/grid/VO/user/file
    SIMBOLIC LINK   ln:/home/user/file or ln:file
    FILE            file:/home/user/file file:/home/user/file2
    HTTPS           https://www.meteo.unican.es/work/WRF4G.tar.gz
    HTTP            http://www.meteo.unican.es/work/WRF4G.tar.gz
    FTP             ftp://www.meteo.unican.es/work/WRF4G.tar.gz
    SFTP            sftp://www.meteo.unican.es/work/WRF4G.tar.gz

host:

Print information about the hosts available on WRF4G.
     
Usage: 
    wrf4g host [ --dbg ] [ list ] [ <hid> ] 
    
Arguments:
    <hid>         Host identifier.

Options:
    --dbg         Debug mode.        
 
Host field information:
    HID           Host identifier.
    ARCH          Architecture.
    JOBS(R/T)     Number of jobs: R = running, T = total.
    LRMS          Local Resource Management System.
    HOSTNAME      Host name. 
    QUEUENAME     Queue name.
    WALLT         Queue wall time.
    CPUT          Queue cpu time.
    MAXR          Max. running jobs.
    MAXQ          Max. queued jobs. 

id:

Manage identities for resources configuring private/public keys and grid credentials.

Usage:  
    wrf4g id <resource> init   [ --dbg ] [ --lifetime=<hours> ]
    wrf4g id <resource> info   [ --dbg ] 
    wrf4g id <resource> delete [ --dbg ] 

 Options:
    -l --lifetime=<hours>   Duration of the identity's lifetime [default: 168].
    --dbg                   Debug mode.
    
Commands:
    init                    Create an identity for a while, by default 168 hours 
                            (1 week). Use the option --lifetime to modify this 
                            value. It adds the configured private key to a ssh-agent
                            and creates a grid proxy using myproxy server.
                            Append the public key to the remote user's 
                            ~/.ssh/authorized_keys file (creating the file, and 
                            directory, if necessary). It tries to load the public 
                            key obtained by appending *.pub to the name of the 
                            configured private key file. Alternative the public 
                            key can be given by public_key variable.
                            It also configures the user's grid certificate 
                            under ~/.globus directory (creating directory, 
                            if necessary) if grid_cert variable is defined.
                                  
    info                    It gives some information about the identity status.
                                
    delete                  The identity is removed from the ssh-agent and the 
                            myproxy server.
Last modified 6 years ago Last modified on Jul 1, 2016 10:27:12 AM