Quantifying forecast uncertainty

Despite the increasing accuracy of weather forecasts, there is an element of uncertainty in all predictions. In 1992, ECMWF pioneered an ensemble prediction system, which now provides a vast range of products to help forecasters deal quantitatively with the day-to-day variations in the predictability of the atmosphere.

ECMWF continues to commit a substantial research effort to the assessment of predictability at all forecast ranges (medium-range, extended and seasonal timescales), and to the investigation of methods to appropriately represent forecast uncertainty.

In any given weather forecast, there are two factors that lead forecast skill to decrease as forecast lead-time increases:

  • Inevitable uncertainties in the initial conditions
  • Necessary approximations in the construction of a numerical model of the real atmospheric system

Both of these cause errors that amplify with time. In fact, these two sources of error cannot be considered independently, since the initial conditions are themselves partly constructed through integration of the underlying numerical model (and thereby include errors resulting from the numerical model) as part of the initialisation process by data assimilation. The ECMWF ensemble forecasting system comprises the Ensemble Prediction System (ENS) for medium-range and extended forecasts, and SEAS5 for seasonal forecasts, which provide estimates of the uncertainty in a forecast.

An ensemble forecast comprises multiple realisations for a single forecast time and location. The different realisations are generated through applying different perturbations to an unperturbed (control) forecast. For the ECMWF medium-range forecasting system, the control forecast is a coarser-resolution realisation of the HRES forecast (the high-resolution forecast). The perturbed forecasts are generated through:

Perturbations to the initial conditions

The ECMWF weather prediction model is run 51 times from slightly different initial conditions. One forecast, called the ENS control forecast, is run from the HRES ECMWF analysis. An additional 50 integrations, the perturbed members, are made from slightly different initial conditions which are designed to represent the uncertainties inherent in the HRES analysis (for more details, see Leutbecher and Palmer, 2008). The initial perturbations are constructed using the singular vector technique and perturbations generated from the ensemble of data assimilations (Buizza et al., 2008).

  • Buizza, R., Leutbecher, M., & Isaksen, L., 2008: Potential use of an ensemble of analyses in the ECMWF Ensemble Prediction System. Q. J. R. Meteorol. Soc., 134, 2051-2066.
  • Leutbecher, M., & Palmer, T. N., 2008: Ensemble forecasting. J. Comp. Phys., 227, 3515-3539.

Perturbations introduced at each model integration

Errors in the underlying numerical model arise from approximations in the discretisations and from assumptions in the parametrizations of unresolved processes. The resulting errors give rise to inherent uncertainty in the accuracy of the model solutions, and in the ECMWF ensemble forecasting system, that uncertainty is simulated by the inclusion of the Stochastically Perturbed Parametrization Tendency scheme (SPPT, Buizza et al., 1999; Palmer et al., 2009; Leutbecher et al. 2017). SPPT represents the uncertainty from the atmospheric physics parametrization schemes by stochastically perturbing the net tendencies (excluding the heating rates due to clear-sky radiation) from the atmospheric physics schemes. Until 2018, the Stochastic Kinetic Energy Backscatter scheme (SKEB, Shutts, 2005; Berner et al., 2009; Leutbecher et al., 2017) was also used to introduce stochastic perturbations to simulate the otherwise unresolved upscale transfer of energy from sub-grid scales to the resolved scales, by introducing a stochastic streamfunction forcing. The most recent model configurations showed SKEB contributing little additional skill (over SPPT), while adding significant computational cost to the ensemble forecasts. As such, SKEB is no longer active in ENS. A detailed discussion of the stochastic parametrizations used at ECMWF and their impact on forecast skill is presented in Leutbecher et al. (2017).

  • Berner, J., Shutts, G. J., Leutbecher, M., & Palmer, T. N., 2009: A spectral stochastic kinetic backscatter scheme and its impact on flow-dependent predictability in the ECMWF ensemble prediction system. J. Atmos. Sci., 66, 603-626.
  • Buizza, R., Miller, M., & Palmer, T. N., 1999: Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Q. J. R. Meteorol. Soc., 125, 2887-2908.
  • Leutbecher, M., Lock, S.-J., Ollinaho, P., Lang, S. T. K., et al. (2017), Stochastic representations of model uncertainties at ECMWF: state of the art and future vision. Q.J.R. Meteorol. Soc, 143: 2315–2339. doi:10.1002/qj.3094
  • Palmer, T. N., Buizza, R., Doblas-Reyes, F., Jung, T., Leutbecher, M., Shutts, G. J., Steinheimer M, & Weisheimer, A., 2009: Stochastic parametrization and model uncertainty. ECMWF Research Department Technical Memorandum. 598, pp. 42
  • Shutts, G. J., 2005: A kinetic energy backscatter algorithm for use in ensemble prediction systems. Q. J. R. Meteorol. Soc., 131, 3079-3102.