# Changes between Version 4 and Version 5 of DRM4G/ResourceConfiguration

Ignore:
Timestamp:
Dec 1, 2016 3:07:55 PM (5 years ago)
Comment:

--

Unmodified
Added
Removed
Modified
• ## DRM4G/ResourceConfiguration

 v4 The configuration file resources.conf is used to describe computing resources. When you start DRM4G, resources.conf file is copied under ~/.drm4g/etc directory by default if it does not exist or under whatever directory specified with **DRM4G_DIR**. The file can be edited directly or by executing the drm4g resource edit command. == Configuration format == The configuration resource file consists of sections, each led by a [section] header, followed by key = value entries. Lines beginning with # are ignored. Permitted sections are [DEFAULT] and [resource_name]. == DEFAULT section == The DEFAULT section provides default values for all other resource sections. == Resource section == Each resource section has to begin with the line [resource_name] followed by key = value entries. === Configuration keys common to all resources: === * username:        Name of the user that will be used to log on to the front-end. * frontend:        Hostname or ip address of either the cluster or grid user interface you'll be connected to. The syntax is "host:port", by default the port used is 22. * frontend:        Hostname or ip address of either the cluster or grid user interface you'll be connected to. The syntax is "''host:port''" and by default the port used is 22. * private_key:     Path to the identity file needed to log on to the front-end. * public key:      Path to the public identity file needed to log on to the front-end. (OPTIONAL, by default the '''private_key''''s value will be taken, to which '''.pub''' will be added) * public key:      Path to the public identity file needed to log on to the front-end. * '''OPTIONAL''': by default the '''private_key''''s value will be taken, to which '''.pub''' will be added) * scratch:         Directory used to store temporary files for jobs during their execution, by default, it is \$HOME/.drm4g/jobs - slurm:         SLURM cluster. - slurm_res:     [http://www.bsc.es/marenostrum-support-services/res RES(Red Española de Supercomputación)] resources. - fedcloud:      it will indicate the DRM4G that this resource will be used to create ''VMs'' (Virtual Machines). [[NoteBox(tip, Note that for communicator you have two options when it comes to accessing a resource through the ssh protocol. If you don't know which one you prefer choose ssh.)]] [[NoteBox(tip, Note that for communicator you have two options when it comes to accessing a resource through the ssh protocol. If you don't know which one you prefer use ssh.)]] === Keys for non-grid resources such as HPC resources: === * queue:             Queue available on the resource. If there are several queues, you have to use a "," as follows "queue  = short,medium,long". * queue:             Queue available on the resource. If there are several queues, you have to use a "," as follows "''queue''  = ''short,medium,long''". * max_jobs_in_queue: Max number of jobs in the queue. * project:           It specifies the project variable and is for TORQUE/PBS,  Grid Engine and LSF clusters. === Keys for grid resources: === * vo:                Virtual Organization (VO) name. * host_filter:       A host list for the VO. Each host is separated by a ",". Here is an example: "host_filter = prod-ce-01.pd.infn.it, creamce2.gina.sara.nl". * host_filter:       A host list for the VO. Each host is separated by a ",". Here is an example: "''host_filter = prod-ce-01.pd.infn.it, creamce2.gina.sara.nl''". * bdii:              It indicates the BDII host to be used. The syntax is "bdii:port". If you do not specify this variable, the LCG_GFAL_INFOSYS environment variable defined on the grid user interface will be used by default. * bdii:              It indicates the BDII host to be used. The syntax is "''bdii'':''port''". If you do not specify this variable, the LCG_GFAL_INFOSYS environment variable defined on the grid user interface will be used by default. * myproxy_server:    Server to store grid credentials. If you do not specify this variable, the MYPROXY_SERVER environment variable defined on the grid user interface will be used by default. === Keys for !FedCloud resources: === '''!FedCloud''' resources are a bit different than the rest. They are not considered to be ''hosts'' so they won't be listed when using the command drm4g host and therefore won't be used to execute jobs. Their only function is to connect to a machine with a cloud proxy certificate ('''X.509 certificates''') capable of giving you access to cloud resources. * vm_communicator:  or  authentication type for the created VMs : - local:       The resource will be accessed directly. - ssh:         The resource will be accessed through ssh's protocol via Paramiko's API. - op_ssh:      The resource will be accessed through OpenSSH's CLI. * vm_user:         Name of the user that will be used to log on to the creates VMs. * bdii:            It indicates the ''BDII'' host to be used. The syntax is "''bdii'':''port''". * You can just use '''''lcg-bdii.cern.ch:2170''''' * myproxy_server:  Server to store cloud credentials. If you do not specify this variable, the MYPROXY_SERVER environment variable defined on the grid user interface will be used by default. * nodes:           It indicates how many VMs you wish to create with the specified configuration * volume:          It's possible to create some extra storage and add it to the VM. With this you can specify how many extra GBs of storage you want. The values of the next configuration keys can be customized at your discretion. A new cloud configuration file has been added to the DRM4G called "'''cloudsetup.json'''" for this reason. This resource keys reference the information saved in this cloud configuration file. * cloud:           Name that describes the site from which the image, that will be used to create the VM, will be acquired. * virtual_image:   It indicates which one of the system images available you will be using * flavour:         It indicates the template of your new VM *  specification Where and how to get the correct values for your cloud configuration file as well as a more in depth explanation of some of these configuration keys can be found in the section [./EGIFedCloud How to configure an EGI FedCloud VM]. == Examples == {{{#!comment #for this maybe another section will be better, since we'll have to explain how to set up the cloud_json and how it works [cesnet] }}} !FedCloud virtual organization {{{ [cesnet_metacloud] enable         = true communicator   = ssh username       = user vo_user        = drm4g_admin vm_communicator= op_ssh vm_user        = drm4g_admin frontend       = ui.macc.unican.es private_key    = ~/.ssh/id_rsa