Changes between Version 58 and Version 59 of udg/ecoms


Ignore:
Timestamp:
Apr 1, 2013 5:32:51 AM (9 years ago)
Author:
gutierjm
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • udg/ecoms

    v58 v59  
    4242This document briefly describes the current state of the data portal, which has initially focused on data from the ''ECMWF's System4 seasonal model'', as agreed in the downscaling parallel session of the kick-off meeting.
    4343
    44 = The THREDDS Data Server = #s.thredds
    45 
    46 The ''SPECS-EUPORIAS Data Portal'' is based on a password-protected `THREDDS data server` (`TDS`) providing metadata and data access to a set of georeferenced atmospheric variables using OPeNDAP and other remote data access protocols. The variables names, units and additional metadata follow the [http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.4/cf-conventions.html CF convention]. The variables are spatial grids based on multidimensional arrays of indexed values, following Unidata's ''_Coordinate convention''[[FootNote(http://www.unidata.ucar.edu/software/netcdf-java/reference/CoordinateAttributes.html)]][[FootNote(http://www.unidata.ucar.edu/software/netcdf-java/tutorial/GridDatatype.html)]].
    47 
    48 Typically the data portal will include information at a daily resolution, but monthly-aggregated values could be also provided in some cases due to data limitations (in particular, ''Mètèo-France'' and ''Met Office'' have agreeed to provide monthly mean hindcasts for their use by the ''SPECS'' and ''EUPORIAS'' partners). In general, the data available will be typical surface variables (e.g. precipitation and near-surface temperature), although several variables (e.g. geopotential and temperature) on pressure levels will also be  stored for the statistical downscaling activities.
    49 
    50 The data gathering activities have initially focused on the ''ECMWF System4 seasonal model''. The Meteorological Archival and Retrieval System (`MARS`) is the main repository of meteorological data at the ''ECMWF'' (European Centre for Medium-Range Weather Forecasts). It contains terabytes of operational and research data as well as data from special projects[[FootNote(http://www.ecmwf.int/services/archive/)]]. The large amount of information stored and the inherent complexities of data access, download and post-processing is a first shortcoming for a flexible use of these datasets by a large number of partners. To overcome this issue, a reduced subset of surface variables[[FootNote(http://www.ecmwf.int/products/changes/system4/technical_description.html#description)]] (precipitation, temperatures and mean sea level pressure) have been downloaded from `MARS` (a colection of `GRIB-1` files) at 0.75º spatial resolution and made available throught the ''SPECS-EUPORIAS data portal''. The downloaded data has been exposed as three different virtual datasets using `TDS`:
    51 
    52 * '''System4 seasonal range (15 members)''': There are twelve initializations (hereafter called `runtimes`) per year (the first of January, February, ...) running for 7 months (hereafter called simply `times`). An ensemble of 15 members is available for the whole 1981-2010 period.
    53 * '''System4 seasonal range (51 members)''': There are only four `runtimes` per year (the first of  February, May, August and November) and the forecasts run for 7 months. An ensemble of 51 members is available for the whole 1981-2010 period.
    54 * '''System4 annual range (15 members)''': As in the previous case, there are four `runtimes` per year, but the forecasts run for 13 months. An ensemble of 15 members is available for the whole 1981-2010 period.
    55 
    56 Data gathering activities will next move to the CFS (http://cfs.ncep.noaa.gov) version 2 hindcast, developed at the ''Environmental Modeling Center at NCEP'' and also to reanalysis and observational datasets.
    5744
    5845
    59 = Accesing the Data Portal via `R` = #s.r.access
    60 
    61 = Accesing the Data Portal via Web = #s.web.access
    62 
    63 
    64 
    65 
    66 = Accessing to the Data portal using Octave = #ex.octave
    67 
    68 [[NoteBox(warn,This section needs revision and integrtion with the [#ex.matlab Matlab example section])]]
    69 
    70 {{{#!text/matlab
    71 >> ver
    72 ----------------------------------------------------------------------
    73 GNU Octave Version 3.6.1
    74 GNU Octave License: GNU General Public License
    75 Operating System: unknown
    76 ----------------------------------------------------------------------
    77 }}}
    78 
    79 {{{#!text/matlab
    80 >> urlwrite('http://www.meteo.unican.es/work/netcdfAll-4.3.jar','netcdfAll-4.3.jar')
    81 >> javaaddpath('./netcdfAll-4.3.jar');
    82 >> javaMethod('setGlobalCredentialsProvider','ucar.nc2.util.net.HTTPSession',javaObject('ucar.nc2.util.net.HTTPBasicProvider','username','password'));
    83 >> ncfile = javaMethod('openDataset','ucar.nc2.dataset.NetcdfDataset','http://www.meteo.unican.es/tds5/dodsC/system4/System4_Seasonal_15Members.ncml');
    84 >> v = ncfile.findVariable('Minimum_temperature_at_2_metres_since_last_24_hours_surface');
    85 >> disp(v.getDimensions.toString)
    86 [   member = 15;,    run = 360;,    time = 215;,    lat = 241;,    lon = 480;]
    87 >> d = v.read('0,11:359:12,31:61,66,475');
    88 >> tmp = javaObject('org.octave.Matrix',d.reduce.copyToNDJavaArray);
    89 >> oldFlag = java_convert_matrix (1);
    90 >> octaveMatrix = tmp.ident(tmp);
    91 [ (30 by 31) array of double ]
    92 >> disp(squeeze(mean(octaveMatrix,2))')
    93  Columns 1 through 13:
    94 
    95    270.79   273.29   271.57   271.04   271.83   272.49   271.48   268.59   271.53   273.82   270.99   274.24   270.99
    96 
    97  Columns 14 through 26:
    98 
    99    271.56   273.99   270.51   272.45   270.66   271.31   272.77   273.44   271.85   273.40   274.16   269.98   271.30
    100 
    101  Columns 27 through 30:
    102 
    103    273.12   271.27   272.29   270.47
    104 }}}
    105 
    106 = Accessing to the Data portal using Matlab= #ex.matlab
    107 
    108 [[NoteBox(warn,This section needs revision])]]
    109 
    110 {{{#!text/matlab
    111 >> ver
    112 -------------------------------------------------------------------------------------
    113 MATLAB Version 7.8.0.347 (R2009a)
    114 MATLAB License Number: 161051
    115 Operating System: Microsoft Windows Vista Version 6.1 (Build 7601: Service Pack 1)
    116 Java VM Version: Java 1.6.0_04-b12 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode
    117 -------------------------------------------------------------------------------------
    118 >> javaaddpath('http://www.meteo.unican.es/work/netcdfAll-4.3.jar');
    119 >> %javaaddpath('ftp://ftp.unidata.ucar.edu/pub/netcdf-java/v4.3/netcdfAll-4.3.jar');
    120 }}}
    121 
    122 {{{#!text/matlab
    123 >> import ucar.nc2.util.net.* %this will download the netcdfAll-4.3.jar
    124 >> HTTPSession.setGlobalCredentialsProvider(HTTPBasicProvider('username','password'));
    125 >> import ucar.nc2.*;
    126 >> import ucar.nc2.dataset.*;
    127 >> ncfile = NetcdfDataset.openDataset('http://www.meteo.unican.es/tds5/dodsC/system4/System4_Seasonal_15Members.ncml');
    128 >> v = ncfile.findVariable('Minimum_temperature_at_2_metres_since_last_24_hours_surface');
    129 >> disp(v.getDimensions)
    130 [   member = 15;,    run = 360;,    time = 215;,    lat = 241;,    lon = 480;]
    131 >> data = v.read('0,11:359:12,31:61,66,475').copyToNDJavaArray();
    132 >> disp(squeeze(mean(data,3)))
    133   Columns 1 through 13
    134 
    135   270.7917  273.2944  271.5666  271.0371  271.8275  272.4928  271.4809  268.5912  271.5313  273.8216  270.9940  274.2363  270.9933
    136 
    137   Columns 14 through 26
    138 
    139   271.5612  273.9899  270.5076  272.4505  270.6556  271.3118  272.7720  273.4359  271.8502  273.3965  274.1638  269.9825  271.3017
    140 
    141   Columns 27 through 30
    142 
    143   273.1195  271.2730  272.2915  270.4669
    144 }}}
    14546
    14647= Example of Data Analysis with `R` = #app1