Version 13 (modified by rodri, 9 years ago) (diff)


[13-Nov-2013]. SPECS Workshop: Verification of ensemble forecasts: A hands-on session. IC3, Barcelon (Spain).

In this workshop, state-of-the-art methods for the evaluation and comparison of ensemble and probability forecasts will be discussed and illustrated. The focus will be on reliability (a.k.a. calibration) as an important property of such forecasts. Verification methods for ensemble forecasts based on fair scores and methods such as the rank histogram will be addressed. To start, some theory of forecast verification and statistical testing will be reviewed. In the main part of the workshop new R routines for comparative ensemble verification, which are built specifically to answer the question "Have the forecasts improved?", will be introduced. Participants are invited to bring their own ensemble data sets and computers to try out the new methods live (the R functions will be made available online). Feedback will be requested to further improve an R package for forecast verification that is being prepared as part of the SPECS project ("Seasonal-to-decadal climate Predictions for the improvement of European Climate Services".

Workshop outline:
Imparted by Stefan Sieger (University of Exeter)

Theory (ca. 1 hour):

  • probabilistic forecast verification
  • ensemble verification
  • fair scores
  • rank histogram
  • statistical testing, bootstrapping

Hands on session (ca. 1 hour):

  • presentation of the R functions
  • live ensemble verification of toy model data
  • live ensemble verification of s2d ensemble forecasts

Discussion (ca. 1 hour):

  • user feedback
  • requests and ideas for more verification methods
  • open questions

Workshop imparted by:

  • Stefan Sieger (University of Exeter).

Attendees from UC:

  • Rodrigo García Manzanas.

Actions taken by UC:

  • Rodrigo has sent an email to Stefan Sieger proposing the collaboration of UC in the development of the R-package for forecast verification (in particular, the implementation of a new routine for drawing ROC curves and computing the associdated ROC Skill Score).


Attachments (1)

Download all attachments as: .zip