WikiPrint - from Polar Technologies

How to configure an EGI FedCloud VM

For configuring an EGI Federated Cloud Virtual Machine, follow the steps below:

First, you'll need to have access to cloud resources. To do that you'll have to follow the instructions given to ?access the federated services.

Then, once you've correctly installed the DRM4G, in your $DRM4G_DIR/etc directory you'll find a file called cloudsetup.json. In it you'll have to define the information needed to create your Virtual Machine(VM).

Configuration format

By default it comes with examples using sites that have been proven to work. The list will be updated with upcoming releases.

This is an example of how the "cloudsetup.json" should look like:

{
    "rocci" : {
        "vo" : "fedcloud.egi.eu",
        "cloud_providers" : {
            "EGI FedCloud - CESNET-METACLOUD" : {
                "endpoint" : "https://carach5.ics.muni.cz:11443",
                "apps" : { 
                    "Ubuntu-14.04" : "http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131"
                },
                "flavours" : {   
                    "Small"  : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#small",
                    "Medium" : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#medium"
                }
            }
        }
    }
}

Cloud configuration keys

How to link the resource configuration file and the cloud setup file

There are a couple of configuration keys in "resources.conf" that reference values found in this JSON.

How to manage VMs

  1. Customize your cloudsetup.json to match your requirements. You can search for the available system images in ?EGI's Applications Database Cloud Marketplace.
  2. Configure a "rocci" resource in resources.conf.
  3. Start the DRM4G and check if the resource has been created successfully :
    [user@mycomputer~]$ drm4g start
    Starting DRM4G .... 
      OK
    [user@mycomputer~]$ drm4g resource list
    RESOURCE            STATE               
    CESNET_Metacloud    enabled
    
  4. Create your remote proxy certificate
    [user@mycomputer~]$ drm4g id CESNET_Metacloud init
    --> Add '/home/user/.ssh/id_rsa' into ssh-agent for 168 hours
     Lifetime set to 7 days, 0:00:00
    --> Copying '/home/user/.ssh/id_rsa' to ~/.ssh/authorized_keys file on 'ui.meteo.unican.es'
    The copy of the public key /home/user/.ssh/id_rsa.pub has been succesfull
     WARNING: It is assumed that the grid certificate has been already configured
    --> Creating '~/.drm4g/security' directory to store the proxy ... 
    Connected (version 2.0, client OpenSSH_5.3)
    Authentication (publickey) successful!
    --> Create a local proxy credential ... 
    Insert your Grid password: 
    Your identity: <user_information>
    Creating proxy ................................................................................... Done
    Proxy Verify OK
    Your proxy is valid until: Thu Dec  8 12:00:00 2016
    Your identity: <user_information>
    Creating proxy .......................................................................................................... Done
    Proxy Verify OK
    A proxy valid for 168 hours (7.0 days) for user <user_information> now exists on myproxy1.egee.cesnet.cz.
    
    
    Warning: your certificate and proxy will expire Thu Dec  8 12:00:00 2016
     which is within the requested lifetime of the proxy
    
    
  5. Create the VM (Virtual Machine) :
  6. Verify that the VM has been created and is been listed as a host :
    [user@mycomputer~]$ drm4g host
    HID ARCH       JOBS(R/T) LRMS       HOSTNAME            
    1   x86_64           0/0 fork       CESNET_Metacloud_XXX.XXX.XXX.XX
    
    
  7. To send it jobs, just submit some as you'd normally do:
    [user@mycomputer~]$ drm4g job submit path/to/job.job --ntasks 10
    ARRAY ID: 0
    
    TASK JOB 
    0    0   
    1    1   
    2    2   
    3    3   
    4    4   
    5    5   
    6    6   
    7    7   
    8    8   
    9    9   
    
  8. And then to check that it is actually doing them:
    [user@mycomputer~]$ drm4g job list
    JID DM   EM   START    END      EXEC    XFER    EXIT NAME            HOST                      
    0   done ---- 12:07:43 12:08:14 0:00:12 0:00:07 0    date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    1   done ---- 12:07:43 12:08:14 0:00:12 0:00:07 0    date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    2   done ---- 12:07:43 12:08:15 0:00:21 0:00:07 0    date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    3   done ---- 12:07:43 12:08:14 0:00:21 0:00:06 0    date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    4   done ---- 12:07:43 12:08:15 0:00:21 0:00:07 0    date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    5   wrap pend 12:07:43 --:--:-- 0:00:04 0:00:01 --   date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    6   wrap pend 12:07:43 --:--:-- 0:00:05 0:00:00 --   date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    7   wrap pend 12:07:43 --:--:-- 0:00:04 0:00:01 --   date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    8   wrap ---- 12:07:43 --:--:-- 0:00:01 0:00:00 --   date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    9   wrap ---- 12:07:43 --:--:-- 0:00:00 0:00:01 --   date.job        CESNET_Metacloud_XXX.XXX.XXX.XX/fork 
    
    
  9. To list resources, you'd usually use the command drm4g resource list, but that doesn't show the created VMs. If you want to see all of them listed, including the created VMs, in case you want to manually connect to it for example, you can use the command option "--all":
    [user@mycomputer~]$ drm4g resource list --all
    Resources:
        CESNET_Metacloud
            communicator:  ssh
            username:      user
            frontend:      ui.meteo.unican.es
            private key:   /home/user/.ssh/id_rsa
            lrms:          rocci
        CESNET_Metacloud_XXX.XXX.XXX.XX
            communicator:  op_ssh
            username:      drm4g_admin
            frontend:      XXX.XXX.XXX.XX
            private key:   /home/user/.ssh/id_rsa
            lrms:          fork
    
  1. Finally, once you're done with your VM, destroy it.

That's it! Now, you can summit jobs to VMs created using the EGI Federated Cloud VO.