wiki:WRF4G2.0/CLI

Version 15 (modified by carlos, 7 years ago) (diff)

--

CLI

wrf4g is a WRF4G's Command Line Interface (CLI) for managing WRF experiments.

Available commands

Type --help option obtain information about all commands:

$ wrf4g --help
WRF4G is a framework for the execution and monitoring of the WRF Modelling System 
in distributed computing resources. For additional information, 
see http://meteo.unican.es/trac/wiki/WRF4G .

Usage: wrf4g [ --version ] [ -h | --help ] [ --dbg ]
             <command> [ <args>... ]

Options:
    -h --help  Show help.
    --version  Show version.
    --dbg      Debug mode.
    
wrf4g commands are:
   start       Start DRM4G daemon, database and ssh-agent
   stop        Stop  DRM4G daemon, database and ssh-agent
   status      Check DRM4G daemon, database and ssh-agent
   conf        Configure DRM4G daemon, scheduler, database and logger parameters
   exp         Manage WRF4G experiments
   rea         Manage WRF4G realizations
   resource    Manage computing resources
   id          Manage reosurce identities
   host        Print information about the hosts
   job         Submit, get status and history and cancel jobs
 
See 'wrf4g <command> --help' for more information on a specific command.

start :

Start DRM4G daemon and ssh-agent. 
    
Usage: 
    wrf4g start [ --dbg ] [ --clear-conf ] [ --disc-jobs ] 
   
Options:
   --dbg                Debug mode.
   --clear-conf         Clear WRF4G's settings stored in .wrf4g directory.
   --disc-jobs          All available jobs on WRF4G will be discared. 

stop:

Stop DRM4G daemon and ssh-agent. 
    
Usage: 
    wrf4g stop [ --dbg ] 
   
Options:
   --dbg    Debug mode.

status:

Check DRM4G daemon and ssh-agent. 
    
Usage: 
    wrf4g status [ --dbg ] 
   
Options:
   --dbg    Debug mode

conf:

Configure DRM4G daemon, scheduler, database and logging parameters.
                
Usage:  
   wrf4g conf ( daemon | sched | logger | database ) [ --dbg ]

Options:
   --dbg    Debug mode

exp :

Manage WRF4G experiments. 
    
Usage: 
    wrf4g exp list                [ --pattern=<name> ]
    wrf4g exp <name> define       [ --dbg ] [ --force ]   [ --from-template=<name> ] [ --dir=<directory> ] 
    wrf4g exp <name> edit         [ --dbg ] 
    wrf4g exp <name> create       [ --dbg ] [ --dry-run ] [ --dir=<directory> ]
    wrf4g exp <name> update       [ --dbg ] [ --dry-run ] 
    wrf4g exp <name> submit       [ --dbg ] [ --dry-run ] [ --priority=<value> ] [ --pattern=<name> ] [ --rea-state=<state> ] [ --rerun ] 
    wrf4g exp <name> status       [ --dbg ] [ --pattern=<name> ] [ --rea-state=<state> ] [ --delay=<seconds> ]  
    wrf4g exp <name> cancel       [ --dbg ] [ --dry-run ] [ --pattern=<name> ] [ --rea-state=<state> ] [ --hard ]
    wrf4g exp <name> set-priority [ --dbg ] [ --dry-run ] [ --pattern=<name> ] <priority>
    wrf4g exp <name> delete       [ --dbg ] [ --dry-run ] 
   
Options:
    --dbg                     Debug mode.
    -n --dry-run              Dry run.
    -f --force                Force to remove if it exists.
    -P --priority=<value>     Fix-priority for scheduling [default: 0]. 
    -p --pattern=<name>       Pattern to find experiments and realizations.
    -s --rea-state=<state>    Select only realizations in the indicated state. Available states :
                              PREPARED, SUBMITTED, RUNNING, PENDING, FAILED and FINISHED 
    -t --from-template=<name> Experiment template, avaible templates are default, single, physics. 
    -d --dir=<directory>      Directory to create or start an experiment [default: ./].
    --delay=<seconds>         Refresh experiment information every delay seconds.
    --rerun                   Force to run although this realization or experiment has finished.
    --hard                    Remove jobs from without synchronizing.
  
Commands:
    list                      Show all the experiments available.
    define                    Create the files needed to define a WRF4G experiment.
    edit                      Edit experiment.wrf4g file.
    create                    Given experiment.wrf4g file, prepare the experiment creating 
                              the realizations and chunks needed.
    update                    Update the experiment configuration.
    submit                    Submit the experiment.
    status                    Check the status of realizations and chunks showing computing resources, 
                              job identifier and exit codes (SEE EXIT CODES) 
    cancel                    Cancel the active realizations by killing their jobs.
    set-priority              Change the scheduling priority of any job releted to a realization. 
                              The priority must be in range [0,20], and the default value is 0. 
                              When a job gets a priority of 20, it becomes an urgent job. This job 
                              is dispatched as soon as possible passing all the scheduling policies.
    delete                    Remove the experiment from the database.

EXIT CODES
    1  : Error creating directory to simulate
    2  : Error creating log directory        
    3  : Error copying apps            
    4  : Error app type does not exist            
    5  : Error executing source script       
    6  : Job already executed  
    7  : Error copying restart files        
    8  : There is a mismatch in the restart date   
    9  : Error copying namelist.wps    
    10 : Error downloading WPS files    
    11 : Error copying boundaries           
    12 : Error modifying namelist
    13 : Error executing PREPROCESSOR
    14 : Error linking GRIB files     
    15 : Error executing UNGRIB
    16 : Error executing METGRID       
    17 : Error executing REAL
    18 : Error uploading WPS files      
    19 : Error executing WRF
    20 : Error executing POSTPROCESSOR 
    21 : Error copying output file    
    22 : Job killed by the system 
    255: Unexpected error

rea:

Manage WRF4G realizations. 
    
Usage: 
     wrf4g rea <name> submit       [ --dbg ] [ --dry-run ] [ --priority=<value> ] [ --rerun ] [ <first_ch> [ <last_ch> ] ]
     wrf4g rea <name> status       [ --dbg ] [ --delay=<seconds> ]
     wrf4g rea <name> log          [ --dbg ] [ --dir=<directory> ] <chunk_id> 
     wrf4g rea <name> set-priority [ --dbg ] [ --dry-run ] <priority>
     wrf4g rea <name> cancel       [ --dbg ] [ --dry-run ] [ --hard ]
   
Options:
    --dbg                 Debug mode.
    -n --dry-run          Dry run.
    --rerun               Force to run although the realization has finished.
    -P --priority=<value> Fix-priority for scheduling [default: 0]. 
    --delay=<seconds>     Refresh experiment information every delay seconds.    
    -d --dir=<directory>  Directory to unpack log files [default: ./].
    --hard                Remove jobs from without synchronizing.
  
Commands:
    submit                Submit the realization.       
    status                Check the status of a realization showing computing resources, 
                          job identifier and exit codes (SEE EXIT CODES).
    log                   Get log files from a chunk.
    set-priority          Change the scheduling priority of any job releted to the realization. 
                          The priority must be in range [0,20], and the default value is 0. 
                          When a job gets a priority of 20, it becomes an urgent job, and it is 
                          dispatched as soon as possible passing all the scheduling policies. 
    cancel                Cancel the realization by killing its jobs.

EXIT CODES
    1  : Error creating directory to simulate
    2  : Error creating log directory        
    3  : Error copying apps            
    4  : Error app type does not exist            
    5  : Error executing source script       
    6  : Job already executed  
    7  : Error copying restart files        
    8  : There is a mismatch in the restart date   
    9  : Error copying namelist.wps    
    10 : Error downloading WPS files    
    11 : Error copying boundaries           
    12 : Error modifying namelist
    13 : Error executing PREPROCESSOR
    14 : Error linking GRIB files     
    15 : Error executing UNGRIB
    16 : Error executing METGRID       
    17 : Error executing REAL
    18 : Error uploading WPS files      
    19 : Error executing WRF
    20 : Error executing POSTPROCESSOR 
    21 : Error copying output file    
    22 : Job killed by the system 
    255: Unexpected error

job :

Submit, get status and history and cancel jobs.

Usage: 
    wrf4g job submit  [ --dbg ] [ --dep <job_id> ... ] <template> 
    wrf4g job list    [ --dbg ] [ --delay=<seconds> ] [ <job_id> ] 
    wrf4g job cancel  [ --dbg ] [ --hard ] <job_id>  
    wrf4g job log     [ --dbg ] <job_id>
    wrf4g job history [ --dbg ] <job_id> 
   
Arguments:
   <job_id>               Job identifier.
   <template>             Job template.

Options:
   --dbg                  Debug mode.
   --dep=<job_id> ...     Define the job dependency list of the job.
   --delay=<seconds>      Refresh experiment information every delay seconds.
   --hard                 Remove jobs from without synchronizing.
    
Commands:
   submit                 Command for submitting jobs.
   list                   Monitor jobs previously submitted.
   cancel                 Cancel jobs.
   log                    Keep track of a job.
   history                Get information about the execution history of a job.

Job field information:
   JID                    Job identification.
   DM                     Dispatch Manager state, one of: 
                                pend, hold, prol, prew, wrap, epil, canl, stop, migr, done, fail.
   EM                     Execution Manager state: pend, susp, actv, fail, done.
   START                  The time the job entered the system.
   END                    The time the job reached a final state (fail or done).
   EXEC                   Total execution time, includes suspension time in the remote queue system.
   XFER                   Total file transfer time, includes stage-in and stage-out phases.
   EXIT                   Job exit code.
   TEMPLATE               Filename of the job template used for this job.
   HOST                   Hostname where the job is being executed.
   HID                    Host identification.
   PROLOG                 Total prolog (file stage-in phase) time.
   WRAPPER                Total wrapper (execution phase) time.
   EPILOG                 Total epilog (file stage-out esphase) time.
   MIGR                   Total migration time.
   REASON                 The reason why the job left this host.
   QUEUE                  Queue name. 

vcp:

Virtual copy a command to copy files using different protocols.
        
Usage:
    wrf4g vcp [ --dbg ] [ --overwrite  ] <source> <dest>

Arguments:

Options:
    --dbg           Debug mode.
    -o --overwrite  If the destination already exists, it will be overwritten.
    
Supported protocols:        
    LFN             lfn:///grid/VO/file  
    GRIDFTP         gridftp://computer:2812/grid/VO/user/file
    RSYNC           rsync://user@computer:34/grid/VO/user/file
    SIMBOLIC LINK   ln:/home/user/file 
    FILE            file:/home/user/file file:/home/user/file2
    HTTPS           https://www.meteo.unican.es/work/WRF4G.tar.gz
    HTTP            http://www.meteo.unican.es/work/WRF4G.tar.gz
    FTP             ftp://www.meteo.unican.es/work/WRF4G.tar.gz
    SFTP            sftp://www.meteo.unican.es/work/WRF4G.tar.gz

resource:

Manage computing resources on WRF4G.
    
Usage: 
    wrf4g resource [ --dbg ] [ list | edit | check ]
    
 Options:
    --dbg     Debug mode.
    
Commands:
    list      Show resources available.    
    edit      Configure resources.
    check     Check out if configured resources are accessible.

host:

Print information about the hosts available on WRF4G.
     
Usage: 
    wrf4g host [ --dbg ] [ list ] [ <hid> ] 
    
Arguments:
    <hid>         Host identifier.

Options:
    --dbg         Debug mode.        
 
Host field information:
    HID           Host identifier.
    ARCH          Architecture.
    JOBS(R/T)     Number of jobs: R = running, T = total.
    LRMS          Local Resource Management System.
    HOSTNAME      Host name. 
    QUEUENAME     Queue name.
    WALLT         Queue wall time.
    CPUT          Queue cpu time.
    MAXR          Max. running jobs.
    MAXQ          Max. queued jobs. 

id:

Manage identities for resources configuring private/public keys and grid credentials.

Usage:  
    wrf4g id <resource> init   [ --dbg ] [ --lifetime=<hours> ]
    wrf4g id <resource> info   [ --dbg ] 
    wrf4g id <resource> delete [ --dbg ] 

 Options:
    -l --lifetime=<hours>   Duration of the identity's lifetime [default: 168].
    --dbg                   Debug mode.
    
Commands:
    init                    Create an identity for a while, by default 168 hours 
                            (1 week). Use the option --lifetime to modify this 
                            value. It adds the configured private key to a ssh-agent
                            and creates a grid proxy using myproxy server.
                            Append the public key to the remote user's 
                            ~/.ssh/authorized_keys file (creating the file, and 
                            directory, if necessary). It tries to load the public 
                            key obtained by appending *.pub to the name of the 
                            configured private key file. Alternative the public 
                            key can be given by public_key variable.
                            It also configures the user's grid certificate 
                            under ~/.globus directory (creating directory, 
                            if necessary) if grid_cert variable is defined.
                                  
    info                    It gives some information about the identity status.
                                
    delete                  The identity is removed from the ssh-agent and the 
                            myproxy server.