{{{#!NewsFlash
'''Recent Changes:'''
[[BlogList(category=Specs)]]
}}}

[[NoteBox(warn,This wiki page and sub-pages are been restructured. Therefore **some links are temporarily broken**. But you can find them at **[wiki:WikiStart wiki index]** or **[search: search them]**)]]

== The ECOMS User Data Gateway ==
The European Climate Observations, Modelling and Services initiative ([http://www.eu-ecoms.eu ECOMS]) coordinates the activities of three ongoing European projects (​[http://www.euporias.eu EUPORIAS], [http://www.specs-fp7.eu SPECS] and [http://naclim.zmaw.de NACLIM]). Different activities carried out in these projects require seasonal forecasts from state-or-the-art forecasting systems (e.g. NCEP/CFSv2 or ECMWF/System4) for a reduced number of variables. This information can be obtained directly from the data providers, but the resulting formats, aggregations and vocabularies may not be homogeneous across datasets, thus requiring some post processing. Moreover, different data policies hold for the various datasets —which are freely available only in some cases— and therefore data access may not be straightforward. Thus, obtaining seasonal climate forecast data is typically a time consuming task.

As part of the data management activities in these projects, the ''ECOMS User Data Gateway'' ('''ECOMS-UDG''') has been developed by the [http://www.meteo.unican.es Santander MetGroup] in order to facilitate seasonal forecasting data access to end users. The needed variables have been downloaded from data providers and locally stored in a THREDDS data server implementing fine-grained user authorization (see the available [wiki:./DataServer/Datasets datasets] and [wiki:./DataServer/ListOfVariables variables]). Thus, users can efficiently retrieve the subsets best suited to their particular research aims (for particular regions, periods and/or ensemble members) from a large volumen of information. Since ​[http://www.r-project.org R] has been adopted for a number of tasks in these projects (validation, downscaling, etc.), an R package for data exploration and access has been developed ([wiki:./RPackage ecomsUDG.Raccess]) and additional functionalities will be developed to meet the users' needs. Moreover, a number of alternative tools are being develop in order to provide different user-friendly interfaces for accessing the information (Python, Matlab, web portals, ... ).

{{{#!comment
As part of the data management activities in these projects, the [http://www.meteo.unican.es Santander Meteorology Group (UC-CSIC)] has established the ''ECOMS User Data Gateway'' ('''ECOMS-UDG''') to provide a single access point to collections of impact-relevant variables, gathered from existing state-of-the-art datasets (see the available [wiki:./DataServer/Datasets datasets] and [wiki:./DataServer/ListOfVariables variables]). The '''ECOMS-UDG''' is composed of a catalogue of datasets and services provided by a [https://www.unidata.ucar.edu/software/thredds/current/tds/TDS.html THREDDS data server] and a suite of tools for user-friendly data exploration, access and post-processing. A fine-grained user authorization system allows different user roles to access data sets with different data policies. Since ​[http://www.r-project.org R] has been adopted for a number of tasks in these projects (validation, downscaling, etc.), an R package for data exploration and access has been developed ([wiki:./RPackage ecomsUDG.Raccess]) and additional functionalities will be developed to meet the users' needs.
}}}

This wiki page provides an up-to-date description of the '''ECOMS-UDG''', including information on the available datasets and variables, and the documentation of the available tools. The following documents and links are the basic references: 

* '''Technical notes:'''  
   * [attachment:DataPortal_Tutorial_v2.pdf  The ECOMS Data Gateway (v2.0)]
   * [attachment:System4Validation_v1.pdf  Validation of System4 dataset (v1.0)]
* '''Tools:'''
   * '''R package''': [wiki:./RPackage#Lateststablereleaseseewhatsnew link to Latest stable release] 
   {{{#!comment
   * `ecomsUDG.Raccess` [https://github.com/SantanderMetGroup/ecomsUDG.Raccess Development version]
   * [raw-attachment:udg.py Python package]
}}}
   * [wiki:EcomsUdg/Interfaces Other interfaces] to the ECOMS-UDG

== Contents of the wiki page:
{{{#!comment
[[TOC(EcomsUdg*,inline,noheading)]]
}}}


1. [wiki:DataServer THREDDS Data Server (TDS)]
   * [wiki:./DataServer/Registration User registration]
   * [wiki:./DataServer/Datasets Available datasets]
   * [wiki:./DataServer/ListOfVariables Available variables]
   * [wiki:./DataServer/Web Exploration via Web]
   * [wiki:./Interfaces Interfaces for Data Access]
   * [wiki:./DataServer/DAP DAP access] 
1. [wiki:./RPackage R Package for Homogeneized Data Access]
   * [wiki:./RPackage/prerequisites Pre-requisites]
   * [wiki:./RPackage/Authentication Authentication]
   * [wiki:./RPackage/Homogeneization Data homogeneization]
   * [wiki:./RPackage/Examples Examples]
{{{#!comment
      * [wiki:./RPackage/Examples/Trellis Trellis maps for data visualization]
      * [wiki:./RPackage/Examples/Drift Analysing model drift]
}}}
{{{#!comment
   * [wiki:./RPackage/History2 History and Versions (for SVN updates)]
   * [wiki:./RPackage/Functions Defined functions]
}}}
1. [wiki:./EndUserNeeds Assessment of Users' Needs]
      * [wiki:./EndUserNeeds/CII EUPORIAS WP22 CIIs]
      * [wiki:./EndUserNeeds/ImpactModels EUPORIAS WP23 impact models]
      * [wiki:./EndUserNeeds/PilotApplications SPECS WP61 pilot applications]
      * [wiki:./EndUserNeeds/ListOfVariables Table of conversion for needed variables - datasets]
      * [wiki:./EndUserNeeds/NoteTimeAggregation A note on the temporal aggregation of climate variables]
1.  [wiki:./FAQs Known Problems and FAQs]
  * [wiki:./FAQs/Technical Technical Issues]
  * [wiki:./FAQs/DataSets Problems with Datasets]
1.  [wiki:./Dissemination Dissemination Activities]


== Issues ==
If you find any issue, problem or make a comment, please [/newticket?component=EcomsUdg drop us a ticket!!]. 

You can submit the ticket anonymously, but if you want to receive notifications on updates and feedbacks you can provide your e-mail address in the ticket form. 

This is the list of tickets, so far:
[[TicketQuery(component=EcomsUdg&type!=spam,format=Table, col=summary|type)]]