wiki:DRM4G/cloud_resource_configuration

Version 1 (modified by minondoa, 5 years ago) (diff)

--

Cloud Resource Configuration

The configuration file resources.conf is used to describe computing resources. When you start DRM4G, resources.conf file is copied under ~/.drm4g/etc directory by default if it does not exist or under whatever directory specified with DRM4G_DIR. The file can be edited directly or by executing the drm4g resource edit command.

Configuration format

The configuration resource file consists of sections, each led by a [section] header, followed by key = value entries. Lines beginning with # are ignored. Permitted sections are [DEFAULT] and [resource_name].

To check how to configure other types of resources check this section?.

Resource section

Each resource section has to begin with the line [resource_name] followed by key = value entries.

Configuration keys for cloud resources:

  • enable: true or false in order to enable or disable a resource.
  • communicator or authentication type :
    • local: The resource will be accessed directly.
    • ssh: The resource will be accessed through ssh's protocol via Paramiko's API.
    • op_ssh: The resource will be accessed through ssh's protocol via OpenSSH's API.
  • username: Name of the user that will be used to log on to the front-end.

  • frontend: Hostname or ip address of either the cluster or grid user interface you'll be connected to. The syntax is "host:port", by default the port used is 22.
  • private_key: Path to the identity file needed to log on to the front-end.
    • This will be used to connect to the machine that'll create the VMs and the Vms themselves.
  • public key: Path to the public identity file needed to log on to the front-end.
    • OPTIONAL, by default a file with the same name as the private_key's plus '.pub' (which is the public key's default name) will be used
  • lrms or Local Resource Management System :
    • fedcloud: this will indicate the DRM4G that this resource will be used to create VMs.
  • max_jobs_running: Max number of running jobs in the queue.
  • vm_communicator: or authentication type for the created VMs :
    • local: The VM will be accessed directly.
    • ssh: The VM will be accessed through ssh's protocol via Paramiko's API.
    • op_ssh: The VM will be accessed through ssh's protocol via OpenSSH's API.
  • vm_user: Name of the user that will be used to log on to the creates VMs.
  • cloud: Name that describes the site from which the image, that will be used to create the VM, will be acquired.
  • bdii: It indicates the BDII host to be used. The syntax is "bdii:port". You can put lcg-bdii.cern.ch:2170 by default
  • myproxy_server: Server to store cloud credentials. If you do not specify this variable, the MYPROXY_SERVER environment variable defined on the grid user interface will be used by default.
  • flavour:
  • virtual_image:
  • nodes:
  • volume:

Note that for communicator you have two options when it comes to accessing a resource through the ssh protocol. If you don't know which one you prefer choose ssh.

For the moment, all VMs will use fork as their lrms.

Cloud configuration section

This is how the cloudsetup.json should look like.

{ 
  "fedcloud" : {
    "vo" : "fedcloud.egi.eu",
    "clouds" : {
      "EGI FedCloud - CESNET-METACLOUD" : {
        "endpoint" : "https://carach5.ics.muni.cz:11443",
        "apps" : { 
          "Ubuntu-14.04" : "http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131"
        },
        "flavours" : {   
          "Small"  : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#small",
          "Medium" : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#medium"
        }
      }
    }
  }
}

From the start it comes with examples of a list of tested working sites. The list will be updated in upcoming releases.

The cloud configuration file consists of pairs of keys and values sections, each led by a [section] header, followed by key = value entries. Lines beginning with # are ignored. Permitted sections are [DEFAULT] and [resource_name].

  • fedcloud:
  • vo:
  • clouds:
  • endpoint:
  • apps:
  • flavours:

Example

EGI FedCloud virtual organization, accessed through a cloud user interface:

[CESNET_Metacloud]
enable            = true
communicator      = op_ssh
username          = user
frontend          = ui.meteo.unican.es
lrms              = fedcloud
vm_communicator   = ssh
vm_user           = drm4g_admin
cloud             = cesnet
bdii              = bdii.grid.sara.nl:2170
myproxy_server    = myproxy1.egee.cesnet.cz
flavour           = Small
virtual_image     = Ubuntu-14.04
nodes             = 1
volume            = 0
max_jobs_running  = 5