Version 54 (modified by juaco, 8 years ago) (diff)


Introduction and usage recommendations

In this section a number of examples for data download and visualization/analysis are presented within the R environment, through the loadECOMS function. Currently, there are four different seasonal to annual hindcasts and one observational gridded dataset available at ECOMS-UDG. All of them are available through the common interface loadECOMS, and therefore the argument values may vary slightly. For instance, arguments members and leadMonth do not apply in the case of the observational gridded dataset WFDEI (see Example 3), and hence ignored. Similarly, the output data structure may vary consequently, and forecast data types include the initialization dates and the names of the chosen members, while this information is not included for other types of gridded data.

The examples given have been kept deliberately simple in order to preserve a moderate output size (<150 Mb) and reasonable execution times (<10 minutes), although larger (and hence more time-consuming) requests can be done. The limitations in data loading depend essentially on two factors:

  1. Object size: Requesting too large objects may deplete the available memory. Currently R runs on 32- and 64-bit operating systems, and most 64-bit OSes (including Linux, Solaris, Windows and OS X) can run either 32- or 64-bit builds of R. The memory limits depends mainly on the build, but for a 32-bit build of R on Windows they also depend on the underlying OS version. For more details, type in the R console help("Memory-limits")
  2. Loading time: The time spent in a request does not depend exclusively on the size of the object to be loaded, but also largely depends on the characteristics of the internet connection and the ECOMS-UDG traffic load at the moment of accessing the data. Thus, if the data request takes too long, we strongly advice to simplify the requested dataset and try to divide the job into smaller queries.

The ecomsUDG.Raccess package has been built using a (Linux) 32-bit OS, as well as all examples and time estimates. Thus, the performance may vary significantly depending on the aforementioned factors. As a rule of thumb, this would be a recommended usage of the loadECOMS function:

  • Point-scale data retrieval: Objects in this case are relatively small because the longitude and latitude dimensions are dropped. As a result, it is possible to load all members (in the case of forecast datasets) for the whole time span (typically around 30 years) without memory problems.
  • Regional-Continental queries: Depending on the size of the spatial window, it may be possible to access all members, but preferably a few members should be selected, and time spans no longer that a decade.
  • Global queries: It is recommended that in this case single-member, single-year selections are done.

Note that this is just an approximate recommendation. Object sizes also depend on the spatial resolution (CFS has approximately 1º res., while System4 is 0.75º and WFDEI 0.5º). Similarly, while GCM data will normally return data for the whole Earth (including oceans) for most variables, many observational datasets (like WFDEI) provide only data for land areas.


The following examples illustrate the data loading process and can serve as a reference to give estimates of time and resulting object size of typical requests: