Version 60 (modified by juaco, 8 years ago) (diff)


Introduction and usage recommendations

In this section a number of examples for data download and visualization/analysis are presented within the R environment, through the loadECOMS function. Currently, there are four different seasonal to annual hindcasts and one observational gridded dataset available at ECOMS-UDG. All of them are available through the common interface loadECOMS, and therefore the argument values may vary slightly. For instance, arguments members and leadMonth do not apply in the case of the observational gridded dataset WFDEI, and hence ignored. Similarly, the output data structure may vary consequently, and forecast data types include the initialization dates and the names of the chosen members, while this information is not included for other types of gridded data.

The examples given have been kept deliberately simple in order to preserve a moderate output size (<150 Mb) and reasonable execution times (<10 minutes), although larger (and hence more time-consuming) requests can be done. The limitations in data loading depend essentially on two factors:

  1. Object size: Requesting too large objects may deplete the available memory. Currently R runs on 32- and 64-bit operating systems, and most 64-bit OSes (including Linux, Solaris, Windows and OS X) can run either 32- or 64-bit builds of R. The memory limits depends mainly on the build, but for a 32-bit build of R on Windows they also depend on the underlying OS version. For more details, type in the R console help("Memory-limits")
  2. Loading time: The time spent in a request does not depend exclusively on the size of the object to be loaded, but also largely depends on the characteristics of the internet connection and the ECOMS-UDG traffic load at the moment of accessing the data. Thus, if the data request takes too long, we strongly advice to simplify the requested dataset and try to divide the job into smaller queries.

Basic loading examples

The ecomsUDG.Raccess package has been built using a (Linux) 32-bit OS, as well as all examples and time estimates. Thus, the performance may vary significantly depending on the aforementioned factors. As a rule of thumb, this would be a recommended usage of the loadECOMS function:

  • EXAMPLE 1: Single Point Selections: Objects in this case are relatively small because the longitude and latitude dimensions are dropped. As a result, it is possible to load all members (in the case of forecast datasets) for the whole time span (typically around 30 years) without memory problems.
  • EXAMPLE 2: Regional-Continental Domain Selections: Depending on the size of the spatial window, it may be possible to access all members, but preferably a few members should be selected, and time spans no longer than one decade.

In the particular case of global domain selections (lonLim and latLim arguments set to NULL) for forecast data, it is recommended that only single-member, single-year selections are performed, due to the large size of this type of requests. Note that this is just an approximate recommendation. Object sizes also depend on the spatial resolution (CFS has approximately 1º horizontal res., while System4 is 0.75º and WFDEI 0.5º). Similarly, while GCM data will normally return data for the whole Earth (including oceans) for most variables, many observational datasets (like WFDEI) provide only data for land areas.

Additional examples