wiki:DRM4G/JobDescription

Job Description

This section will explain how to create a template with which to configure DRM4G's jobs.

Syntax

<VARIABLE> = ["]<VALUE>["]

Template options

  • NAME: Name of the job (filename of the Job Template by default).
  • EXECUTABLE: The executable file. If the executable is a shell command, you MUST specify the absolute path. For instance, /usr/bin/python, /bin/bash, /usr/bin/perl, etc.
  • ARGUMENTS: Arguments to the above executable.
  • NP: The number of processors to be allocated to a job.
  • INPUT_FILES: A comma-separated pair of local/remote filenames. If the remote filename is missing, the local filename will be preserved in the execution host.
  • OUTPUT_FILES: A comma-separated pair of remote/local filenames. If the local filename is missing, the remote filename will be preserved in the client host.
  • STDIN_FILE: Standard input file.
  • STDOUT_FILE: Standard output file. By default is stdout.${JOB_ID}.
  • STDERR_FILE: Standard error file. By default is stderr.${JOB_ID}.
  • REQUIREMENTS: A boolean expression evaluated for each available host, if the evaluation returns true the host will be considered to submit the job.
  • ENVIRONMENT: User defined, comma-separated environment variables.

Examples

  • How to define a template to execute an R script:
    NAME = My_R_script
    EXECUTABLE  = /usr/bin/R -f example.r
    STDOUT_FILE = stdout_file.${JOB_ID}
    STDERR_FILE = stderr_file.${JOB_ID}
    INPUT_FILES = example.r
    OUTPUT_FILES = output.data
    
  • How to define a template to execute a Python script:
    NAME = My_Python_script
    EXECUTABLE  = example.py
    STDOUT_FILE = stdout_file.${JOB_ID}
    STDERR_FILE = stderr_file.${JOB_ID}
    INPUT_FILES = example.py
    
    NAME = My_Python_script
    EXECUTABLE  = /usr/bin/python example.py
    STDOUT_FILE = stdout_file.${JOB_ID}
    STDERR_FILE = stderr_file.${JOB_ID}
    INPUT_FILES = example.py
    

Requirement Expressions

Syntax

The syntax of the requirement expressions is defined as:

stmt::= expr
expr::= VARIABLE '=' INTEGER
         | VARIABLE '>' INTEGER
         | VARIABLE '<' INTEGER
         | VARIABLE '=' STRING
         | expr '&' expr
         | expr '|' expr
         | '!' expr
         | '(' expr ')'

Each expression is evaluated to 1 (True) or 0 (False). Only those hosts for which the requirement expression is evaluated to True will be considered to execute the job.

Logical operators are as expected ( less '<', greater '>', '&' AND, '|' OR, '!' NOT), '=' means equals with integers. When you use '=' operator with strings, it performs a shell wild-card pattern matching.

Variables

The REQUIREMENTS values are:

  • HOSTNAME: Execution host (e.g. mycomputer).
  • ARCH:Architecture of the execution host (e.g. i686, x86_64).
  • LRMS_TYPE: Type of local DRM system for execution (e.g. pbs, sge).
  • QUEUE_NAME: Name of the queue (e.g. default, short).
  • QUEUE_MAXTIME:Maximum wall time of jobs in the queue.
  • QUEUE_MAXCPUTIME:Maximum CPU time of jobs in the queue.
  • QUEUE_MAXCOUNT:Maximum count of jobs that can be submitted in one request to the queue.
  • QUEUE_MAXRUNNINGJOBS:Maximum number of running jobs in the queue.
  • QUEUE_MAXJOBSINQUEUE:Maximum number of queued jobs in the queue.

Examples

REQUIREMENTS = LRMS_TYPE = "pbs"                   # Only use pbs
REQUIREMENTS = HOSTNAME = "*.es"                   # Only hosts ending in ".es"
REQUIREMENTS = HOSTNAME = "mycomputer"             # Only use mycomputer
REQUIREMENTS = ARCH = "x86_64"                     # Only host x86_64 architecture 
REQUIREMENTS = ARCH = "x86_64" & HOSTNAME = "*.es" # Only hosts ending in ".es" and have x86_64 architecture

Environment Expressions

Syntax

The syntax of the environment expressions is specified in a comma-separated, source/destination pair:

stmt::= VARIABLE = VALUE, VARIABLE = VALUE, ...

Variables

The variables defined in the ENVIRONMENT are:

  • PPN: Specify the number of processors per node requested for the job.
  • CPUTIME:Maximum amount of CPU time used by all processes in the job (HH:MM:SS).
  • WALLTIME:Maximum amount of real time during which the job can be in the running state (HH:MM:SS).
  • MEMORY:Maximum amount of physical memory used by the job (MB).

Examples

ENVIRONMENT = WALLTIME = 00:01:00                # 60 seconds of max walltime
ENVIRONMENT = WALLTIME = 00:01:00, MEMORY = 2000 # 60 seconds of max walltime and 2 GB of RAM memory
Last modified 4 years ago Last modified on Apr 10, 2017 1:56:47 PM