wiki:DRM4G/ResourceConfiguration/EGIFedCloud

Version 3 (modified by minondoa, 5 years ago) (diff)

--

How to configure an EGI FedCloud VM

For configuring an EGI Federated Cloud Virtual Machine, follow the steps below:

First, you'll need to have access to cloud resources. To do that you'll have to follow the instructions given to access the federated services.

Then, once you've correctly installed the DRM4G, in your $DRM4G_DIR/etc directory you'll find a file called cloudsetup.json. In it you'll have to define the information needed to create your virtual machine (VM).

Configuration format

  • It's a JSON file, so it's syntax includes the following:
    • Data is represented in name/value pairs.
    • Curly braces hold objects and each name is followed by ':'(colon), the name/value pairs are separated by "," (commas).

By default it comes with examples using sites that have been proven to work. The list will be updated with upcoming releases.

This is an example of how the "cloudsetup.json" should look like:

{
    "fedcloud" : {
        "vo" : "fedcloud.egi.eu",
        "clouds" : {
            "EGI FedCloud - CESNET-METACLOUD" : {
                "endpoint" : "https://carach5.ics.muni.cz:11443",
                "apps" : { 
                    "Ubuntu-14.04" : "http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131"
                },
                "flavours" : {   
                    "Small"  : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#small",
                    "Medium" : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#medium"
                }
            }
        }
    }
}

Cloud configuration keys

  • fedcloud: Indicates the name of the Virtual Organization (VO) being used. In the future more may be added.
    • vo: it's used for authorization, to get the proxy certificate
      • fedcloud.egi.eu: shouldn't be changed for FedCloud sites
    • clouds: it will usually contain a list of sites with the VM configurations.
      • <site_name>: name that represents the site where your VM will be hosted
        • endpoint: end of a communication channel, URL of their server
        • apps: list of system images
          • <system_image_name>: OCCI ID that represents the system image you'll be using
        • flavours: list of template IDs
          • <hardware_template>: template ID that defines hardware specifications, sizes for RAM, disk and number of CPU cores

How to link the resource configuration file and the cloud setup file

There are a couple of configuration keys in "resources.conf" that reference values found in this JSON.

  • From the resource configuration file "resources.conf" you have to indicate which "cloud", "app" and "flavour" you want to use from all of the ones you have defined in the cloud setup file.
    • It is done with the "cloud", "virtual_image" and "flavour" keys from the resource configuration file respectively.
  • You can modify the name (not the value) of the "<site_name>", "<system_image_name>" and "<hardware_template>", so be sure to write the same value on the resource configuration file as the name on the cloud setup file.
    • The value in "resources.conf" has to match with the name in "cloudsetup.josn"
      [CESNET_Metacloud]
      cloud             = EGI FedCloud - CESNET-METACLOUD
      flavour           = Small
      virtual_image     = Ubuntu-14.04
      
      "clouds" : {
          "EGI FedCloud - CESNET-METACLOUD" : {
              "endpoint" : "https://carach5.ics.muni.cz:11443",
              "apps" : { 
                  "Ubuntu-14.04" : "http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131"
               },
              "flavours" : {   
                  "Small"  : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#small",
                  "Medium" : "http://fedcloud.egi.eu/occi/compute/flavour/1.0#medium"
              }
          } 
      }
      
  1. Configure the esrVO resource. If the grid user interface has defined LCG_GFAL_INFOSYS and MYPROXY_SERVER variables, you do not have to indicate bdii and myproxy_server keys in your configuration:
    [user@mycomputer~]$ drm4g resource edit
    [esrVO]
    enable         = true
    communicator   = local
    username       = user
    frontend       = ui.meteo.unican.es
    lrms           = cream
    vo             = esr
    bdii           = bdii.grid.sara.nl:2170
    myproxy_server = px.grid.sara.nl
    
  2. List and check if resource has been created successfully :
    [user@mycomputer~]$ drm4g resource list
    RESOURCE            STATE               
    esrVO               enabled
    
  3. List the CEs available on the esr VO:
    [user@mycomputer~]$ drm4g host list
    HID ARCH       JOBS(R/T) LRMS       HOST
    0   x86_64           0/0 cream-pbs  esrVO::cream.afroditi.hellasgrid.gr
    1   x86_64           0/0 cream-pbs  esrVO::ce1.ipgp.fr    
    2   x86_64           0/0 cream-pbs  esrVO::cr1.ipp.acad.bg
    3   x86_64           0/0 cream-pbs  esrVO::sbgce2.in2p3.fr
    4   x86_64           0/0 cream-pbs  esrVO::ce0.bordeaux.inra.fr
    5   x86_64           0/0 cream-pbs  esrVO::cce.ihep.ac.cn 
    6   x86_64           0/0 cream-pbs  esrVO::ce02.ngcc.acad.bg
    7   x86_64           0/0 cream-pbs  esrVO::ce01.macc.unican.es
    8   x86_64           0/0 cream-pbs  esrVO::cygnus.grid.rug.nl
    9   x86_64           0/0 cream-pbs  esrVO::t2ce06.physics.ox.ac.uk
    10  x86_64           0/0 cream-lsf  esrVO::ce1.ts.infn.it 
    11  x86_64           0/0 cream-lsf  esrVO::gridce1.pi.infn.it
    12  x86_64           0/0 cream-lsf  esrVO::gridce3.pi.infn.it
    13  x86_64           0/0 cream-pbs  esrVO::cream02.grid.uoi.gr
    14  x86_64           0/0 cream-pbs  esrVO::lapp-ce02.in2p3.fr
    15  x86_64           0/0 cream-pbs  esrVO::grid002.jet.efda.org
    16  x86_64           0/0 cream-lsf  esrVO::gridce4.pi.infn.it
    17  x86_64           0/0 cream-lsf  esrVO::gridce0.pi.infn.it
    18  x86_64           0/0 cream-lsf  esrVO::gridce2.pi.infn.it
    19  x86_64           0/0 cream-pbs  esrVO::t2ce06.physics.ox.ac.uk
    20  x86_64           0/0 cream-pbs  esrVO::grid0.fe.infn.it
    21  x86_64           0/0 cream-pbs  esrVO::ce0.m3pec.u-bordeaux1.fr
    22  x86_64           0/0 cream-pbs  esrVO::juk.nikhef.nl  
    23  x86_64           0/0 cream-pbs  esrVO::gridce.ilc.cnr.it
    24  x86_64           0/0 cream-lsf  esrVO::cert-37.pd.infn.it
    25  x86_64           0/0 cream-pbs  esrVO::cream-ce-2.ba.infn.it
    26  x86_64           0/0 cream-sge  esrVO::cccreamceli09.in2p3.fr
    27  x86_64           0/0 cream-sge  esrVO::cccreamceli10.in2p3.fr
    28  x86_64           0/0 cream-pbs  esrVO::gazon.nikhef.nl
    29  x86_64           0/0 cream-pbs  esrVO::klomp.nikhef.nl
    30  x86_64           0/0 cream-pbs  esrVO::cream-ce-3.ba.infn.it
    31  x86_64           0/0 cream-pbs  esrVO::cream-ce-4.ba.infn.it
    32  x86_64           0/0 cream-pbs  esrVO::creamce.gina.sara.nl
    33  x86_64           0/0 cream-lsf  esrVO::prod-ce-01.pd.infn.it
    34  x86_64           0/0 cream-pbs  esrVO::creamce2.gina.sara.nl
    35  x86_64           0/0 cream-pbs  esrVO::creamce3.gina.sara.nl
    36  x86_64           0/0 cream-slur esrVO::ce3.ui.savba.sk
    37  x86_64           0/0 cream-pbs  esrVO::glite-cream.scai.fraunhofer.de
    38  x86_64           0/0 cream-pbs  esrVO::cream-ce02.marie.hellasgrid.gr
    39  x86_64           0/0 cream-pbs  esrVO::cream-ce01.marie.hellasgrid.gr
    40  x86_64           0/0 cream-pbs  esrVO::fal-pygrid-44.lancs.ac.uk
    41  x86_64           0/0 cream-pbs  esrVO::hepgrid6.ph.liv.ac.uk
    42  x86_64           0/0 cream-pbs  esrVO::cream-ce01.ariagni.hellasgrid.gr
    43  x86_64           0/0 cream-pbs  esrVO::snf-189278.vm.okeanos.grnet.gr
    44  x86_64           0/0 cream-pbs  esrVO::snf-458754.vm.okeanos.grnet.gr
    45  x86_64           0/0 cream-pbs  esrVO::hepgrid5.ph.liv.ac.uk
    46  x86_64           0/0 cream-pbs  esrVO::cream01.kallisto.hellasgrid.gr
    47  x86_64           0/0 cream-pbs  esrVO::hepgrid10.ph.liv.ac.uk
    48  x86_64           0/0 cream-pbs  esrVO::hepgrid97.ph.liv.ac.uk
    
  4. Create an identity for 7 days:
    [user@mycomputer~]$ drm4g id esrVO init
    --> Create a local proxy credential ... 
    Insert your Grid password: 
    Your identity: /DC=es/DC=irisgrid/O=unican/CN=user
    Creating proxy ........................................................................................... Done
    Proxy Verify OK
    Your proxy is valid until: Thu Feb 26 21:37:19 2015
    Your identity: /DC=es/DC=irisgrid/O=unican/CN=user
    Creating proxy .................................................................... Done
    Proxy Verify OK
    A proxy valid for 168 hours (7.0 days) for user /DC=es/DC=irisgrid/O=unican/CN=user now exists on px.grid.sara.nl.
    
  5. Check the timeleft of your identity:
    [user@mycomputer~]$ drm4g id esrVO info 
    --> Grid credentials
    subject  : /DC=es/DC=irisgrid/O=unican/CN=user/CN=proxy/CN=proxy
    issuer   : /DC=es/DC=irisgrid/O=unican/CN=user/CN=proxy
    identity : /DC=es/DC=irisgrid/O=unican/CN=user
    type     : full legacy globus proxy
    strength : 2048 bits
    path     : /home/user/.drm4g/security/px.grid.sara.nl
    timeleft : 167:57:52  (7.0 days)
    

That's it! Now, you can summit jobs to the esr VO. Keep in mind that you will have to renew your identity depending on the proxy-lifetime used.

Attachments (5)

Download all attachments as: .zip