Changes between Version 56 and Version 57 of udg/ecoms

Apr 1, 2013 5:29:32 AM (9 years ago)



  • udg/ecoms

    v56 v57  
    63 The ''SPECS-EUPORIAS Data Portal'' can be accessed through the '''Data Portal URL''' provided in the abstract. First of all, an authentication dialog will request a valid user name and password.
    65 [[Image(loginTHREDDS.png,align=center,width=320px,title=Authentication dialog)]]
    67 {{{#!comment
    68 \begin{figure}[H]
    69 \begin{center}
    70 \includegraphics[width = 0.8 \linewidth]{loginTHREDDS.png}
    71 \caption{Authentication dialog}
    72 \label{fig:login}
    73 \end{center}
    74 \end{figure}
    75 }}}
    77 Afterwards, the different datasets described in [#s.thredds TDS section] are listed as links in the web browser window. 
    79 [[Image(fig01.png,align=center,width=320px,title=Catalog of the EUPORIAS-SPECS System4 datasets. Note that although they only include a few variables, their size range from one to four Terabytes)]]
    81 {{{#!comment
    82 \begin{figure}[H]
    83 \begin{center}
    84 \includegraphics[width = 0.85 \linewidth]{fig01.png}
    85 \caption{Catalog of the EUPORIAS-SPECS System4 datasets. Note that although they only include a few variables, their size range from one to four Terabytes.
    86 }
    87 \label{fig:dir}
    88 \end{center}
    89 \end{figure}
    90 }}}
    93 By clicking in any of the datasets, a new window will appear providing information on the variables and geospatial and time coverages, and offering different options for data access and/or visualization.
    95 [[Image(fig02.png,align=center,width=320px,title=Detail of a particular dataset with information on the included variables and geospatial and time coverages. The different options for data access and visualization are also shown.)]]
    97 {{{#!comment
    98 \begin{figure}[H]
    99 \begin{center}
    100 \includegraphics[width=0.8 \linewidth]{fig02.png}
    101 \caption{Detail of a particular dataset with information on the included variables and geospatial and time coverages. The different options for data access and visualization are also shown.}
    102 \label{fig:mainwin}
    103 \end{center}
    104 \end{figure}
    105 }}}
    107 Currently, only the `OPeNDAP` access service is fully operative in the portal. Therefore, in this example, we will illustrate the use of this service, which allows selecting time/spatial data slices from the `OPeNDAP` data access form shown in figure and downloading the resulting data in both ''ASCII'' and ''Binary'' formats.
    109 [[Image(openDAPwindow.png,align=center,width=320px,title=Detail of the OPeNDAP dataset access form for a particular dataset.)]]
    111 {{{#!comment
    112 \begin{figure}[H]
    113 \begin{center}
    114 \includegraphics[width=0.8 \linewidth]{openDAPwindow.png}
    115 \caption{Detail of the OPeNDAP dataset access form for a particular dataset.}
    116 \label{fig:opendapwin}
    117 \end{center}
    118 \end{figure}
    119 }}}
    121 Note that, as explained before, the variables provided by the data portal (e.g. minimum temperature) are stored as gridsets. Thus, in addition to these variables, also auxiliary coordinate variables (lat, lon, run, time, member) should be handled for geo-temporal data referencing ([attachment:openDAPwindow.png see Figure]). Moreover, three time coordinates are included as referece for different grid variables because they are defined for different forecast times (one extra time for precipitation and different temporal resolution for mean sea level pressure). Note that this highly complicates the direct analysis of the data and, hence, this options is only recommend for data exploration. In the following we show how to use this service to explore the structure of the datasets and to obtain simple pieces of information in `ASCII format`.
    123 By default, if no specifications are given in the different subsetting boxes of the OpenDAP form, the whole data on the whole spatio/temporal and member ranges of the dataset would be accessed. However, this option will raise an error due to the large size of the request (the maximum size of a single request has been set to 100 Mbytes in the ''SPECS-EUPORIAS data portal'' for the sake of multi-connection efficiency). The basic steps to retrieve subsets of data are the following:
    125  1. To select a variable click on the checkbox to its left.
    126  1. To constrain the variable, edit the information that appears in the text boxes below the variable. This is a   vector of integers indicating index positions of length three, with the following order: `[start:stride:end]`.
    127  1. To get ''ASCII'' or ''binary'' values for the selected variables, click on the ''Get ASCII'' or ''Get Binary'' buttons of the ''Action'' field. Note that the URL displayed in the ''Data URL'' field is updated as you select and/or constrain variables. The URL in this field can be cut and pasted in various `OPeNDAP` clients.
    129 The main disadvantage of the `OPeNDAP` service from the end-user point of view is that the specifications for subsetting dimensions are not given in their original magnitudes (i.e., latitudes and longitudes are not given in decimal degrees), but by the indexes of their position along their respective axes (note that first index value is always 0). Thus, to find out the indexes for the desired selection, we need to dump and analyze the particular values defined in the coordinate variable. For instance, this figure shows the 241 values defined for the `lat` (latitude) coordinate, as provided by the ''Get ASCII'' option (selecting the corresponding check-box).
    131 [[Image(latlonDump.png,align=center,width=320px,title=Text file displaying the values for the lat (latitude) coordinate variable.)]]
    133 {{{#!comment
    134 \begin{figure}[H]
    135 \begin{center}
    136 \includegraphics[width=\linewidth]{latlonDump.png}
    137 \caption{Text file displaying the values for the \texttt{lat} (latitude) coordinate variable.}
    138 \label{fig:latlonDump}
    139 \end{center}
    140 \end{figure}
    141 }}}
    143 Using these facilities it can be obtained after some calculations that the closest `lat` and `lon` coordinates for a particular location of interest (e.g. `Madrid`) are 66 and 475, respectively. Thus, the time series for Madrid corresponding to the example described in the previous section (minimum temperature forecasts for January with one-month lead time, i.e. from the simulations started the first of December) could be requested as shown in Figure
    145 [[Image(opendapquery.png,align=center,width=320px,title=Detail of the query from the OPeNDAP dataset access form to retrieve a subset (a time series for a single gridbox) of minimum temperature.)]]
    147 {{{#!comment
    148 \begin{figure}[H]
    149 \begin{center}
    150 \includegraphics[width= 0.95 \linewidth]{opendapquery.png}
    151 \caption{Detail of the query from the OPeNDAP dataset access form to retrieve a subset (a time series for a single gridbox) of minimum temperature.}
    152 \label{fig:opendapquery}
    153 \end{center}
    154 \end{figure}
    155 }}}
    158 Note that the indices selected for the run coordinate correspond to the December initilizations (index positions 11, 23,...; note that indexes start in 0) and for the time coordinate correspond to January (positions, 31 to 62, in days after the run time). Note that the proper use of this service requires a full understanding of the data structure and, therefore, it is only advised for data exploration.
    160 = Accessing to the Data portal using Python (Pydap version) = #ex.pydap
    162 [[NoteBox(warn,This section needs revision)]]
    164 {{{#!csh
    165 [user@host ~]$ pip install Pydap
    166 ........................................................................
    167 [user@host ~]$ python
    168 Python 2.7.2 (default, Mar  3 2012, 10:45:44)
    169 [GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
    170 Type "help", "copyright", "credits" or "license" for more information.
    171 >>>
    172 }}}
    174 {{{#!python
    175 >>> from pydap.client import open_url
    176 >>> dataset = open_url('')
    177 >>> print type(dataset)
    178 <class 'pydap.model.DatasetType'>
    179 >>> print dataset.keys()
    180 ['lat', 'lon', 'run', 'time', 'time1', 'time2', 'member', 'Maximum_temperature_at_2_metres_since_last_24_hours_surface', 'Minimum_temperature_at_2_metres_since_last_24_hours_surface', 'Mean_temperature_at_2_metres_since_last_24_hours_surface', 'Total_precipitation_surface', 'Mean_sea_level_pressure_surface']
    181 >>> MN2T24 = dataset['Minimum_temperature_at_2_metres_since_last_24_hours_surface']
    182 >>> print MN2T24.dimensions
    183 ('member', 'run', 'time', 'lat', 'lon')
    184 >>> print MN2T24.shape
    185 (15, 360, 215, 241, 480)
    186 >>> arr = MN2T24[0,11:360:12,31:62,66,475]
    187 >>> print numpy.squeeze(numpy.mean(arr,2))
    188 [ 270.79171753  273.29437256  271.56661987  271.03707886  271.82745361
    189   272.49279785  271.48086548  268.59121704  271.53125     273.82156372
    190   270.99401855  274.23626709  270.99328613  271.56115723  273.98986816
    191   270.50756836  272.45046997  270.65560913  271.31182861  272.77200317
    192   273.4359436   271.85021973  273.39648438  274.16384888  269.98248291
    193   271.30166626  273.11950684  271.27301025  272.29147339  270.46688843]
    194 }}}
    19665= Accessing to the Data portal using Octave = #ex.octave