Work

Publications Lectures Selected articles Current research

Roberto Buizza

European Centre for Medium-Range Weather Forecasts

This article was published in the ECMWF Newsletter # 92 (Autumn 2001).

• Weather-related damages
• The ECMWF Ensemble Prediction System (EPS)
• Quality of the EPS probabilistic precipitation predictions
• EPS-value sensitivity to ensemble size and resolution
• EPS precipitation prediction of two flooding events over Italy
• Risk management with the EPS
• References

Damage due to the December 1999 French storm is estimated to be about $10bn (Cornford 2000). The average 1955-1999 annual damage over the US due to tornado, hurricane and flood stands at about $12bn (Extreme Weather Sourcebook 2001, see Pielke & Downton 2000, Pielke & Landsea 1998). Insurance companies have paid out $91.8bn in losses from weather-related natural disasters in the period 1990-1998 (Dunn & Flavin, 1999). In the United States, average costs of $16 billion are incurred annually for weather-related damages (Pielke 1997). Weather-related damages due to severe weather events have been increasing in the past 10 years, and there is an increasing request for weather information (data, forecasts) to manage weather-risk exposures.

Single deterministic, categorical forecasts may fail to predict the intensity, the spatial location or the temporal occurrence of severe weather events. A more complete approach to numerical weather prediction is to estimate the time evolution of the probability density function of forecast states. The ECMWF Ensemble Prediction System (EPS) based on a finite number of numerical integrations is a practical tool that can be used to estimate the time evolution of the probability density function of forecast states.

Attention in this work is focused on intense precipitation events that can cause severe damages, and on the potential use of EPS probabilistic predictions for risk assessment. Two main issues are discussed:

• What is the skill of EPS probabilistic precipitation predictions?
• How can EPS probabilistic predictions be used to assess weather-related risks?

The ECMWF Ensemble Prediction System (EPS)

At ECMWF, the EPS became part of the daily operational forecasting system on the 19^th of December 1992. It started with 33 members run with spectral triangular truncation T63 and 19 vertical levels (Molteni et al 1996). From the 11^th of December 1996 to the 21^st of November 2000 the EPS had been based on 51 members at T_L159L40 resolution (spectral triangular truncation T159 with linear grid, equivalent to a grid spacing of about 120km at mid latitudes). On the 21^st of November 2000, the EPS resolution had been increased to T_L255L40, which is equivalent to a spatial grid spacing of about 80km at mid latitudes (Buizza et al 2001).

The EPS includes a scheme to simulate also model uncertainties due to random model error in the parameterised physical processes (Buizza et al. 1999a). For each day d, the 50 perturbed initial conditions are defined by adding initial perturbations to the operational (T_L511L60) analysis, interpolated to the EPS resolution. The day d initial perturbations are defined using the singular vectors (Buizza & Palmer 1995) growing in the forecast range between day d and day d+2 at initial time, and the singular vectors growning between day d-2 and day d at final time (Barkmeijer et al 1999). The initial perturbations are scaled to have local comparable to analysis error estimates.

The quality of a forecasting system is described by the statistical characteristics of the joint distribution of forecasts and observations (Katz & Murphy 1997). This implies that many scalar measures are required to describe the EPS forecast quality, or in other words that each measure describe a particular aspect of the correspondence between forecasts and observations. For reason of space, this work presents only a limited set of results: the reader is referred to the list of referred works (and references therein) for more complete analyses of the quality of EPS forecasts.

One of the most commonly used measures of the quality of probabilistic forecasts is the Brier score (Brier 1950). The Brier score measures the mean-square error of probabilistic forecasts: it ranges from 1 for a perfect forecast to 0 for an unskilful one. The Brier skill score is defined as the fractional improvement in the Brier score of the EPS forecasts over forecasts based on the climatological distribution.

Figure 1a and Figure 1b show the Brier skill score of the EPS probabilistic prediction of 1, 10, 20 and 50 mm/d of precipitation over the United States during the cold (November through March) and warm (May through September) seasons (Mullen & Buizza, 2001a). Results are based on precipitation forecasts of the (old) 51*T_L159L31 EPS during three years, 1997 through 1999, with EPS forecasts interpolated on a regular latitude-longitude 1.25° degree grid, verified against River Forecast Centers data averaged on the same grid (this data-set includes approximately 5000 stations reporting 24-h accumulated precipitation valid at 12 UTC). Figure 1a and Figure 1b show that accuracy is higher in winter than in summer, with skill depending on the precipitation threshold. EPS forecasts of precipitation in excess of 50 mm/d show no skill, while EPS forecasts of more than 20 mm/d are skilful up to forecast day 6 in winter and 2 in summer. Precipitation is more predictable in winter than in summer presumably because the synoptic forcing is stronger and convection is less prevalent. Mullen & Buizza (2001a) investigated also the capacity of EPS forecasts to discriminate precipitation events using the area under a Relative Operating Characteristics curve as quality measure, and concluded that according to this measure forecasts for all four thresholds were skilful in both seasons.

Figure 1. Winter (a) and summer (b) Brier skill score for the EPS probabilistic prediction of 1 (blue solid), 10 (red dashed), 20 (green dotted) and 50 (black chain-dashed)mm/d (from Mullen & Buizza 2001a).

EPS precipitation verification over Australia for a 3-year period against observations confirms these results (E. Ebert, 2001, personal communication). A similar verification has not yet been performed over Europe because of the lack of unique data-set grouping observations from a high-density network over such an area. To overcome this problem, Buizza et al. (1999b) used the 0-24h ECMWF high-resolution forecast as a proxy for verification. Their conclusions were qualitatively in line with the results of Mullen & Buizza (2001a) and Ebert (2001, personal communication). The comparison of the performance of the old and the new high-resolution (T_L255L40) ensemble system for 87-cases indicates that forecasts from the new high-resolution EPS are more accurate (Mullen & Buizza 2001b). Results indicate a gain in predictability between 12-36 hours.

Ensemble size and model resolution are two key parameters that define the configuration of an ensemble prediction. On the one hand, it is desirable to have fine resolution in physical space so that the model is able to simulate events with the spatial scale of interest, and on the other hand it is desirable to have a fine resolution in probability space to sample the tails of the forecast probability distribution function. Given that computation resources available for operational weather prediction are limited, it is necessary to find the right balance between model spatial resolution and ensemble size.

Mullen & Buizza (2001b) have evaluated the sensitivity of the EPS performance to ensemble size and resolution in terms of potential economic value estimated using a static cost-loss decision model for a dichotomous event (Thompson 1952; Wilks 1995). According to this type of decision models, a decision maker can chose to pay a cost C to protect against a possible loss L (with L>C): if protective action is not taken, than the decision maker incurs a loss L if the adverse event incurs.

Figure 2 shows four cost-loss potential economic value curves for the day-5 prediction of the event "precipitation in excess of 20 mm/d" for 57 winter cases, with EPS forecasts verified at rain-gauge sites (Mullen & Buizza, 2001b). Three curves show the potential economic value for forecasting systems that require approximately the same amount of CPU (central processor unit) time: a 51-member T_L159L31 ensemble, a 15-member T_L255L31 ensemble and a single T_L319L31 forecast. The forth curve refers to a 51-member T_L255L31 ensemble. Figure 2 shows that if CPU-cost is not an issue, increasing the horizontal resolution improves the potential value. But if CPU-cost matters, ensemble size is more important than resolution especially for small cost-loss ratio. Thus, if the potential economic value of the 20 mm/d precipitation prediction is the most important quality measure used to define the optimal ensemble configuration, than a large-size low-resolution ensemble system is to be preferred to a small-size high-resolution one. It should be stressed that these results are based on raw forecast probabilities defined as the number of ensemble members predicting the event divided by the total membership; the sensitivity to ensemble size may change if distribution-fitting is applied to the EPS probability forecasts. The reader is referred to Mullen & Buizza (2001b) and Richardson (2001) for a more detailed discussion of these and related issues.

Figure 2. Cost/Loss Value curves for day-5 EPS precipitation forecasts of more than 20 mm/d over the US verified at rain gauge sites, averaged for 57 winter cases for 4 forecasting systems: 51*TL159L31 (solid blue), 51*TL255L31 (solid red), 15*TL255L31 (solid red with diamonds) and single TL319L31 (dashed green) (from Mullen & Buizza 2001b).

In November 1994 and in October 2001 intense precipitation caused severe flooding over Northern Italy. Both events occurred during autumn, a period of the year when over the Mediterranean the water vapour content of near-surface air masses may still be high due to the relatively high sea surface temperatures, and over a region where local orographic forcing acts to reinforce local ascent.

Very intense precipitation hit Piemonte in Northern Italy (45° N, 8° E) between 5 and 6 November 1994. The single deterministic forecast provided by the EPS control proved to be very accurate (not shown). Figure 3 shows the 51*T_L255L40 (re-run) EPS probability of "24h precipitation in excess of 50mm" verifying between 12 UTC of the 5^th and the 6^th of November, predicted on the 29^th and on the 31^st of October, and on the 1^st of November. Results indicate a very consistent signal, with probability values increasing from 2-30% for the longest lead-time to 60-100% for the shortest lead-time over the region where more than 50mm is observed. These probability forecasts supported the deterministic forecast.

Figure 3. Piemonte 1994. (a) EPS forecasts for the event "24h precipitation in excess of 50 mm", started on the 29^th of October and valid between 12 UTC of the 5^th and 12 UTC of the 6^th of November (t+168h to 192h). (b) as (a) but for the forecasts started on the 31^st of October (t+120h to t+144h). (c) as (a) but for the forecasts started on the 2^nd of November (t+72h to t+96h). (d) proxy for verification, defined as the 24h T_L319L31 forecast started on the 5^th of November. Contour isolines are 2, 5, 25, 50 and 75% for probabilities, and 2, 10, 25 and 50 mm/d for precipitation.

Heavy and prolonged precipitation over the catchment area of the Po river (North-western Italy) associated by a cut-off low over the Mediterranean caused severe damage between 14 and 16 October 2000. The single deterministic forecast provided by the EPS control issued on successive days proved rather inconsistent and failed to predict more than 50mm of rainfall 96 hours before the event (not shown). Figure 4 shows the 51*T_L255L40 EPS probability of "48h precipitation in excess of 50mm" verifying between 12 UTC of the 14^th and the 16^th of October, predicted on the 8^th, the 10^th and the 12^th of October. EPS probability forecasts show a westward shift the area of maximum probability as the lead-time decreases, with the 48-to-96 hour forecast indicating a 30-60% probability of more than 50mm/48h over the region where this amount is observed. In this case, EPS forecasts proved to be essential in assessing the possibility of precipitation in excess of 50mm.

Figure 4. Piemonte 2000. (a) EPS forecasts for the event "48h precipitation in excess of 50 mm", started on the 8^th and valid between 12 UTC of the 14^th and 12 UTC of the 16^th of October (t+144h to 192h). (b) as (a) but for the forecasts started on the 10^th (t+96h to t+144h). (c) as (a) but for the forecasts started on the 12^th (t+48h to t+96h). (d) proxy for verification, defined as the 48h T_L319L31 forecast started on the 14^th. Contour isolines are 2, 5, 25, 50 and 75% for probabilities, and 2, 10, 25 and 50 mm/d for precipitation.

EPS precipitation forecasts have been used to predict the average precipitation over the Po’ river catchment area. Figure 5 (P. Viterbo, 2001, personal communication) shows the average accumulated precipitation over the Po’ river catchment area predicted by the EPS started on the 7^th of October. The average accumulated precipitation given by 24h T_L511L60 forecast can be used as a proxy for the observed value. Figure 5 shows that the proxy for observation is included in the EPS forecast range. The EPS gives an 8% (26%) probability of a 10-day total accumulated precipitation over the catchment area of more than 100mm (75mm).

Figure 5. Accumulated precipitation averaged over the Po’ river catchment area predicted on 7 October 2001 by the EPS perturbed members (red thin solid), the EPS control forecast (blue solid) and the T_L511L60 forecast (black solid). The green-solid line shows a proxy for verification given by the accumulated precipitation given by subsequent 24h T_L511L60 forecasts (P. Viterbo, 2001, personal communcation).

Early indications of severe weather events are necessary to improve the quality of systems designed to issue early warnings of potentially severe damages (Buizza & Hollingsworth 2001a,b). Timely precipitation forecasts are necessary to drive hydrological models used in flooding forecasting.

Single deterministic forecasts predict only one possible future scenario, say the most probable one, while ensemble systems based on multiple integration can be used to estimate the whole probability distribution function of forecast states. Some users may, in fact, be more interested to know what is the probability of a rare event to occur or not occur rather than to know what is the most likely scenario.

Given an individual characterised by a weather-dependant utility U, weather forecasts from each member of an ensemble system can be transformed into a forecast of the probability distribution of utility. Figure 6 is a schematic of the flow of information from forecasts generated by an ensemble system to the utility probability density function estimated using model M. Ensemble forecasts can be used to up-date and refine an a-priory estimate of possible losses estimated using climatology (green) and to quantify the probability that a maximum-acceptable-loss L_MAX can occur.

Figure 6. Schematic of information flow from ensemble forecasts to utility distribution function. EPS single-membed weather forecasts (top boxes) can be used as an input to utility models to predict the probability distribution function of utilities (bottom).

Taylor & Buizza (2001) followed this approach to predict energy demand using ensemble predictions of surface wind, temperature and cloud cover, and a model M that translated each weather state into one energy demand scenario. They demonstrated that errors in energy demand prediction can be reduced by using ensemble forecasts. Similar approaches can be followed by using static/dynamic decision models to find the optimal sequence of actions that minimise weather-related losses (Richardson 2000, Johnson & Holt 1997, Smith et al 2001).

Barkmeijer, J., Buizza, R., & Palmer, T. N., 1999: 3D-Var Hessian singular vectors and their potential use in the ECMWF Ensemble Prediction System. Q. J. R. Meteor. Soc., 125, 2333-2351.

Brier, G. W., 1950: Verification of weather forecasts. Mon. Wea. Rev., 78, 1-3.

Buizza, R., & Hollingsworth, A., 2001a: Storm prediction over Europe using the ECMWF Ensemble Prediction System. Meteorol. Appl., in press.

Buizza, R., & Hollingsworth, A., 2001a: Severe weather prediction using the ECMWF EPS: the European storms of December 1999. ECMWF Newsletter No. 89, available at: ECMWF, Shinfield Park, Reading RG2 9AX, UK, 2-12.

Buizza, R., Miller, M., & Palmer, T. N., 1999a: Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Q. J. R. Meteorol. Soc., 125, 2887-2908.

Buizza, R., Hollingsworth, A., Lalaurette, F., & Ghelli, A., 1999b: Probabilistic predictions of precipitation using the ECMWF Ensemble Prediction System. Weather and Forecasting, 14, 2, 168-189.

Buizza, R., Richardson, D. S., & Palmer, T. N., 2001: The new 80-km high-resolution ECMWF EPS. ECMWF Newsletter No. 90, available at: ECMWF, Shinfield Park, Reading RG2 9AX, UK, 2-9.

Cornford, S. G., 2000: Human and economic impacts of weather events in 1999, WMO Bull., 49, 356-375.

Dunn, S. & Flavin, C., 1999: Destructive storms drive insurance losses up: will taxpayers have to bail out insurance industry?. WorldWatch Press release, available at http:/www.worldwatch.org/alerts/990325.html).

Johnson, S. R., & Holt, M. T., 1995: The value of weather information. In Katz, W., & Murphy, A. H., Economic value of weather and climate forecasts, Cambridge University Press, UK, 75-107.

Katz, W., & Murphy, A. H., 1997: Economic value of weather and climate forecasts. Cambridge University Press, UK, pp222 (ISBN 0-521-43420-3).

Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T., 1996: The ECMWF Ensemble Prediction System: methodology and validation. Q. J. R. Meteorol. Soc., 122, 73-119.

Molteni, F., Buizza, R., Marsigli, C., Montani, A., Roberto Buizzaozzi, F., & Paccagnella, T., 2001. A strategy for high-resolution ensemble prediction. Part I: definition of representative members and global-model experiments. Q. J. R. Meteorol. Soc., 127, 2069-2094.

Mullen, S., & Buizza, R., 2001a: Quantitative precipitation forecasts over the United States by the ECMWF Ensemble Prediction System. Mon. Wea. Rev.,129, 638-663.

Mullen, S., & Buizza, R., 2001b: The Impact of horizontal resolution and ensemble size on probabilistic forecasts of precipitation by the ECMWF Ensemble Prediction System. Mon. Wea. Rev., in press.

Pielke, R. A., Jr., et al., 1997: Workshop on the Social and Economic Impacts of Weather, April 2-4. Available at http://www.esig.ucar.edu/socasp/weather1.

Pielke, R. A., Jr., & Landsea, C. W., 1998: Normalized hurricane damages in the United States: 1925-1995. Weather. And Forecasting, 13, 621-631.

Pielke, R. A., Jr., & Downton, M., 2000: Precipitation and damaging floods: trends in the United States, 1932-1997. J. of Climate, 13, 3625-3637.

Richardson, D. R., 2000: Skill and economic value of the ECMWF ensemble prediction system, Q. J. R. Meteorol. Soc., 126, 649-668.

Richardson, D. S., 2001: Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size. Q. J. R. Meteorol. Soc., 127, to appear.

Smith, L. A., Roulstone, M. S., & von Hardenberg, J., 2001: End-to-end ensemble forecasting: towards evaluating the economic value of the Ensemble Prediction System. ECMWF Tech. Memorandum No. 336, available at: ECMWF, Shinfield Park, Reading RG2 9AX, UK.

Taylor, J., & Buizza, R., 2001: Energy demand prediction using the ECMWF Ensemble Prediction System. Intern. J. Forecast., in press. Also available as RD-Technical Memorandum No. 312, ECMWF, Shinfield Park, Reading RG2 9AX, U.K.

Thompson, J. C., 1952: On the operational deficiencies in categorical weather forecasts. Bull. Amer. Meteorol. Soc., 33, 223-226.

Wilks, D. S., 1995: Statistical methods in the atmospheric sciences. Academic Press Inc., San Diego, pp 4 67 (ISBN 0-12-751965-3)

(back to top of the page)