A preliminary assessment of the Stream 2 seasonal hindcasts (all start dates) is available from this page. The assessment has been carried out for the period 1960-2005.

The ECMWF (dark blue), Météo-France (dark green), INGV (orange), IfM (light blue) and HadGEM2 (pink) ensembles all have nine members. The DePreSys_PP (grey) ensemble has been constructed from the nine different versions of HadCM3 computing the anomalies separately for each version. The 45-member multi-model puts together the anomalies of the ECMWF, Météo-France, INGV, IfM and HadGEM2 ensembles. The nine-member multi-model (yellow) takes randomly nine members out of the 45 available from the multi-model, taking at least one member per single-model experiment. It should allow to assess the impact of the ensemble size in the multi-model ensemble.

The results are for the four start dates available in Stream 2 (1st of February, May, August and November) and for the forecast periods 1 (first month), 2-4 and 5-7 months.

The html pages contain a large amount of plots and uploading the pages might take some time. Please, be patient. The plots have a reasonable size, but they can be zoomed in by clicking on them.

Climatology maps for ECMWF, Météo-France, INGV, IfM and HadGEM2. ERA40/OPS are also included as reference. The maps include the mean climate, the climatological standard deviation, the mean bias and the ratio of standard devations with respect to the reference. Climatologies for versions 0, 1, 2, 3, 4, 5, 6, 7 and 8 of DePreSys_PP are also available. Please, bear in mind that DePreSys_PP uses flux corrections, so the mean temperatures over the oceans are expected to follow closely the observations.
Spatial maps of scores for ECMWF, Météo-France, INGV, IfM, HadGEM2, DePreSys_PP, a nine-member multi-model and the 45-member multi-model: anomaly correlation coefficient, perfect-model anomaly correlation coefficient, spread (normalized using the observed interannual standard deviation), mean square skill score and ratio between the spread (measured in terms of standard deviation around the ensemble mean) and the RMSE for near-surface temperature, precipitation, mean sea level pressure, 500 hPa geopotential height and 850 hPa temperature. Significance tests will be available next.
Sumary of deterministic scores for selected regions: as above, but the scores have been computed over a number of regions of interest. Variances/covariances have been estimated for each grid point and then averaged before computing the correlations or ratios. The horizontal axis corresponds to the start date and the forecast period, with the results for a given start date separated from the others using vertical dotted lines. Confidence intervals will be available next.
Time series for ECMWF, Météo-France, INGV, IfM, HadGEM2, DePreSys_PP, a nine-member multi-model and the 45-member multi-model: The green box-and-whisker shows the range of the ensemble values, the blue dot the ensemble mean and the red dot the reference (GPCP for precipitation and ERA40/OPS for the rest). The horizontal lines correspond to the lower and upper tercile of the hindcast (blue) and reference (red) climatological distributions. Some scores (SNR for signal/to/noise ratio) are given on the top right corner, with the p values in brackets. Please note that the vertical scales are computed automatically.
Summary of scores for time series: ensemble-mean correlation, ratio between the spread (measured in terms of standard deviation around the ensemble mean) and the RMSE, ranked probability skill score and debiased ranked probability skill score as a function of the start date and forecast period. Some of the plots show some dots that correspond to the p-value for the score to be significantly different from zero (not applicable to the ratio spread/RMSE). The scale for the p value can be found on the right side of the plot.
Summary of probabilistic scores: Brier skill score (BSS), BSS for infinitely ensemble-sized forecasts (BSI), reliability skill score (REL), resolution skill score (RES) and ROC skill score (ROC) over a number of regions of interest. Contingency tables have been averaged over the grid points of the region before computing the corresponding skill score. The horizontal axis corresponds to the start date and the forecast period, with the results for a given start date separated from the others using vertical dotted lines.

18.03.2009