wiki:WRF4G2.0/AdvancedConfiguration

Version 19 (modified by carlos, 7 years ago) (diff)

--

Advanced Configuration

Optional environment variables

WRF4G uses the following environmental variables:

  • WRF4G_DIR: Directory for local configuration. If it is undefined, $WRF4G_DIR will be ~/.wrf4g.
  • EDITOR: Select the editor to configure configuration files. vi editor is used by default.

Logging configuration

Edit by typing wrf4g conf logger.

Configuration file for WRF4G logging. For more information about logging configuration directives, you should check out python logging configuration. Additionally, %(WRF4G_DIR)s variable indicates where WRF4G configuration resides.

[loggers]
keys=root,drm4gIm,drm4gEm,drm4gTm,drm4gConfigure,drm4gManager,drm4gCommunicator

[handlers]
keys=handDrm4gIm,handDrm4gEm,handDrm4gTm,handConfigure,handManager,handCommunicator

[formatters]
keys=form01

[logger_root]
handlers=

[logger_drm4gIm]
handlers=handDrm4gIm
level=INFO
qualname=drm4g.core.im_mad

[logger_drm4gEm]
handlers=handDrm4gEm
level=INFO
qualname=drm4g.core.em_mad

[logger_drm4gTm]
handlers=handDrm4gTm
level=INFO
qualname=drm4g.core.tm_mad

[logger_drm4gConfigure]
handlers=handConfigure
level=INFO
qualname=drm4g.core.configure

[logger_drm4gManager]
handlers=handManager
level=INFO
qualname=drm4g.managers

[logger_drm4gCommunicator]
handlers=handCommunicator
level=INFO
qualname=drm4g.communicators

[handler_handDrm4gIm]
class=handlers.RotatingFileHandler
level=INFO
formatter=form01
args=('%(WRF4G_DIR)s/var/log/drm4g_im.log','w',5000000,4)

[handler_handDrm4gEm]
class=handlers.RotatingFileHandler
level=INFO
formatter=form01
args=('%(WRF4G_DIR)s/var/log/drm4g_em.log','w',5000000,4)

[handler_handDrm4gTm]
class=handlers.RotatingFileHandler
level=INFO
formatter=form01
args=('%(WRF4G_DIR)s/var/log/drm4g_tm.log','w',5000000,4)

[handler_handConfigure]
class=handlers.RotatingFileHandler
level=INFO
formatter=form01
args=('%(WRF4G_DIR)s/var/log/drm4g_configure.log','w',5000000,4)

[handler_handManager]
class=handlers.RotatingFileHandler
level=INFO
formatter=form01
args=('%(WRF4G_DIR)s/var/log/drm4g_manager.log','w',5000000,4)

[handler_handCommunicator]
class=handlers.RotatingFileHandler
level=INFO
formatter=form01
args=('%(WRF4G_DIR)s/var/log/drm4g_communicator.log','w',5000000,4)

[formatter_form01]
format=%(asctime)s %(levelname)-9s %(name)-8s %(message)s

Database configuration access

Edit by typing wrf4g conf database.

URL = mysql+pymysql://<user>:<password>@<host>:<port>/<dbname>

Default configuration :

[DEFAULT]
URL = mysql+pymysql://wrf4guser:Meteo2011@localhost:25000/WRF4GDB

Daemon configuration

Edit by typing wrf4g conf daemon.

General configuration:

  • GWD_PORT: Daemon port.( Default value is 6725 )
  • MAX_NUMBER_OF_CLIENTS: Number of connections. (Default value is 50 )

Size of pools:

  • NUMBER_OF_ARRAYS: The number of array-jobs that will be handled by the scheduler. (Default value is 200000)
  • NUMBER_OF_JOBS: The number of jobs that will be handled by the scheduler. (Default value is 200000)
  • NUMBER_OF_HOSTS: The number of hosts that will be handled by the scheduler. (Default value is 1000)
  • NUMBER_OF_USERS: The number of different users. (Default value is 30 )

Intervals:

  • SCHEDULING_INTERVAL: Period between two scheduling actions. (Default is 16 seconds)
  • DISCOVERY_INTERVAL: How often the information manager searches for new hosts. (Default is 15 seconds)
  • MONITORING_INTERVAL: How often the information manager updates the information of each resource. (Default is 20 seconds)
  • POLL_INTERVAL: How often the underlying grid middleware is query about job state. (Default is 1000 seconds)

Scheduler configuration

Edit by typing wrf4g conf sched.

General configuration:

  • DISPATCH_CHUNK: The maximum number of jobs that will be dispatched each scheduling action. Default value is 100, 0 dispatch as many jobs as possible.
  • MAX_RUNNING_USER: The maximum number of simultaneous running jobs per user. Default value is 0, 0 to dispatch as many jobs as possible.

Job priority configuration. Pending jobs are prioritize according four policies:

  • Fixed Priority (FP) Policy. Assign a fixed priority to each job (FP range [0,19]):
    • FP_WEIGHT: Weight for the policy. Default is 1 (real numbers allowed).
    • FP_USER[<username>]: Priority for jobs owned by <username>. Use the special username DEFAULT to set default priorities.
    • FP_GROUP[<groupname>]: Priority for jobs owned by users in group <groupname>.
  • Share (SH) Policy. Allows to establish a dispatch ratio among users. It tracks the jobs submitted by each user so it targets the defined ratio:
    • SH_WEIGHT: Weight for the policy. Default is 1 (real numbers allowed).
    • SH_USER[<username>]: Share for jobs owned by <username>. Use the special username DEFAULT to set default shares.
    • SH_WINDOW_DEPTH: Number of intervals (windows) to "remember" each user dispatching history. The submissions of each window are exponentially "forgotten". Default is 5, and the maximum value is 10.
    • SH_WINDOW_SIZE: The size of each interval in days. Default is 1, real numbers allowed.
  • Waiting-time (WT) Policy. The priority of a job is increased linearly with the waiting time to prevent job starvation:
    • WT_WEIGHT: Weight for the policy. Default is 1 (real numbers allowed)
  • Deadline (DL) Policy. The priority of a job is increased exponentially as its deadline approaches:
    • DL_WEIGHT: Weight for the policy. Default is 1 (real numbers allowed)
    • DL_HALF: Number of remaining days when the job should get half of the maximum priority. Default is 1 (real numbers allowed).