3 | | '''Attendees:''' |
4 | | * Met Office: Jemma Gornall, Karina Williams, Pete Falloon, Carlo Buontempo, Alberto Arribas and Rutger Dankers. |
5 | | * University of Leeds: Kathryn Nicklin. |
6 | | * WUR: Ronald Hutjes. |
7 | | * SMHI: Ilias Pechlivanidis. |
8 | | * University of Cantabria: Rodrigo García Manzanas. |
| 3 | In this workshop, state-of-the-art methods for the evaluation and |
| 4 | comparison of ensemble and probability forecasts will be discussed |
| 5 | and illustrated. The focus will be on reliability (a.k.a. |
| 6 | calibration) as an important property of such forecasts. |
| 7 | Verification methods for ensemble forecasts based on fair scores |
| 8 | and methods such as the rank histogram will be addressed. To |
| 9 | start, some theory of forecast verification and statistical |
| 10 | testing will be reviewed. In the main part of the workshop new R |
| 11 | routines for comparative ensemble verification, which are built |
| 12 | specifically to answer the question "Have the forecasts |
| 13 | improved?", will be introduced. Participants are invited to bring |
| 14 | their own ensemble data sets and computers to try out the new |
| 15 | methods live (the R functions will be made available online). |
| 16 | Feedback will be requested to further improve an R package for |
| 17 | forecast verification that is being prepared as part of the SPECS |
| 18 | project ("Seasonal-to-decadal climate Predictions for the |
| 19 | improvement of European Climate Services". |
18 | | '''Open discussions:''' |
19 | | * Possibility to establish new case studies out of Europe, based on the skill of seasonal forecasts. |
| 31 | Hands on session (ca. 1 hour): |
| 32 | * presentation of the R functions |
| 33 | * live ensemble verification of toy model data |
| 34 | * live ensemble verification of s2d ensemble forecasts |
| 35 | |
| 36 | Discussion (ca. 1 hour): |
| 37 | * user feedback |
| 38 | * requests and ideas for more verification methods |
| 39 | * open questions |
| 40 | |
| 41 | '''Workshop imparted by:''' |
| 42 | * Stefan Sieger (University of Exeter). |
| 43 | |
| 44 | '''Attendees from UC:''' |
| 45 | * Rodrigo García Manzanas. |
| 46 | |
| 47 | '''Actions taken by UC:''' |
| 48 | * Rodrigo has sent an email to Stefan Sieger proposing the collaboration of UC in the development of the R-package for forecast verification (in particular, the implementation of a new routine for drawing ROC curves and computing the associdated ROC Skill Score). |
| 49 | |
| 50 | '''Links:''' |
| 51 | * [https://github.com/sieste SPECS R-package for forecast verification] |