Roberto Buizza

European Centre for Medium-Range Weather Forecasts.

(Extract from the Encyclopaedia of Atmospheric Sciences, Academic Press, 2002, in press)

• Introduction
• Numerical Weather Prediction
• Sources of forecast errors
• Chaotic behaviour of the atmosphere
• The ensemble approach to numerical weather prediction
• Simulation of initial uncertainties
• Simulation of model errors
• Operational applications of ensemble prediction
• Validations of ensemble systems
• Conclusions
• Further reading

The atmosphere is a complex dynamical system with many degrees of freedom. In numerical weather prediction, the state of the atmosphere is described by the spatial distribution of wind, temperature, specific humidity, liquid water content and surface pressure. The mathematical differential equations used to predict the system time evolution include Newton's laws of motion and the laws of thermodynamics. Numerical weather prediction models predict the time evolution of the atmospheric state by solving numerically the system equations.

A deterministic forecast is a single integration of the system equations. The practical usefulness of a single deterministic weather forecast is limited by the day-to-day variability in its accuracy. This variability is partly associated with fluctuations in the predictability of the atmospheric flow, with predictable states (i.e. flows characterized by a slow amplification of initial errors) alternated by unpredictable states (i.e. flows characterized by a fast amplification of initial errors).

Ensemble systems are practical tools designed to assess the predictability of the daily atmospheric flow. More generally, they can be used to predict the time evolution of the probability density function (PDF) of forecast states. They can be used, for example, to predict the probability of intense rainfall or cold temperatures over the Euro-Atlantic region (Figure 1).

Ensemble systems should be designed to simulate the effect of all sources of forecast errors. In particular, they should simulate the effect of uncertainties in the knowledge of the initial state of the system and the effects of the approximations made in numerical weather prediction models.

Ensemble systems have been operational since 1992 at the European Centre for Medium-Range Weather Forecasts (ECMWF, UK) and at the National Centers for Environmental Prediction (NCEP, US), and since 1995 at the Canadian Meteorological Center (CMC, Canada). These three ensemble systems have been designed to estimate the forecast PDF in the short- and medium-forecast range, i.e. for up to 14 days. Beside this operational activity, many international centers, universities, national and regional meteorological centers have been involved in research and experimental activities in this field. It should also be mentioned that experimental ensemble systems are currently under development and are tested for seasonal time scales.

Despite that fact that most of the examples and of the discussions reported hereafter are based on results obtained during the past years by the operational medium-range ensemble systems, most of the discussions can be applied to seasonal ensemble systems.

Numerical weather prediction

In numerical models, the state of the atmosphere is described for a finite number of vertical levels and at a series of grid points by a set of state variables such as temperature T, velocity v, specific humidity and cloud liquid content q and surface pressure p. In other words, the state vector y of the system is defined by (T,v,q,p) for all vertical levels and all grid points. The phase space of the system is the N-dimensional space defined by the (T,v,q,p) coordinates.

The system attractor is defined by the set of past, present and future atmospheric states y. For each time t, a unique point on the system attractor identifies the state of the atmosphere. The time evolution of atmospheric states during subsequent times t between the initial time t₀ and the final time t₁, t₀<t<t₁, is represented by the set of points that described the system orbit between t₀ and t₁ (Figure 2).

Figure 2. Schematic of a 2-dimensional section of the phase space of the atmospheric system. The phase-space of the system is the set of past, present and future atmospheric states. The time evolution of the system between the initial time T₀ and the time T₁ is represented by the dotted line connecting the points y(T₀) and y(T₁). This line is called the orbit of the system.

The mathematical differential equations describing the atmospheric motions include Newton’s laws of motion in the form ‘acceleration equals force divided by mass’, of conservation of mass and energy, and the laws of thermodynamics. These equations, written in terms of the state variables y, can be approximated with a set of algebraic difference equations for the tendencies (i.e. for the variations in time) of the state variables.

These equations include parameterization schemes that simulate the effects of physical processes such as radiative transfer, turbulent mixing, orographic forcing and moist processes. The parameterization of these physical processes is probably one of the most difficult and uncertain area of weather modeling.

Schematically, the equations of motion defined in terms of the state variables y=(T,v,q,p) can be written as

(Eq. 1)

where A and P identify, respectively, the contribution to the full equation tendency of the non-parameterized and parameterized physical processes. The time integration of Equation [1] from the initial time t₀ to the forecast time t

(Eq. 2)

describes the time evolution of the atmospheric flow from the initial state y(t₀) to the final state.

The initial state of the system y(t₀) is defined by observed weather variables. Denote by o(τ) the vector of all meteorological observations made in a time interval 2*D centered at time t₀, t₀-D<τ<t₀+D. The initial state of the system y(t₀) is defined in such a way that, for all times τ with t₀-D<τ<t₀+D, y(τ) is closest to o(τ) Practically, y(t₀) is computed by minimizing a cost function J(y(t₀))

(Eq. 3)

which is the sum of the distance d_B(..,..) of y(t₀) from the background field y_B(t₀) and the time-integrated distance d_O(..,..) of y(t) and the observation vector o(t). (Note that the two distances are different, since one is defined using the covariance matrix of background errors and the other the covariance matrix of observation errors.) The minimum of the cost function can be considered as the best estimate of the true state of the atmosphere. This computational process is referred to as data assimilation. Note that the accuracy of a data assimilation procedure depends on the accuracy/approximations of the atmospheric model used to compute y(τ) starting from t₀-D,

(Eq. 4)

Moreover, in order to compare the two vectors y and o, the atmospheric model is used to transform observed variables into atmospheric state vector variables (i.e. to go from the observations’ phase-space to the model variables’ phase-space).

Some of the observations, such as the ones from weather balloons or radiosondes, are taken at specific times at fixed locations. Others, such as the ones from airplanes, ships or satellites, are not fixed in space. Generally speaking, there is a great variability in the density of the observation network, with data over oceanic regions, in particular, characterized by very coarse resolution. Observations cannot be used directly to start model integrations, but they must be modified in a dynamically consistent way to obtain a suitable data set.

The fact that, at any time t₀, only a limited number (limited with respect to the degrees of freedom of the system) of observations are available and that part of the globe is characterized by a poor coverage, introduces uncertainties in the initial conditions. Observational errors, usually in the smallest scales, amplify and through non-linear interactions spread to the large scales and eventually affect the skill of these latter ones. The presence of uncertainties in the initial conditions is one of the sources of forecast errors.

A second source of forecast error is related to the intrinsic approximations made in the numerical models of the atmospheric system. A requirement for skilful predictions is that numerical models are able to accurately simulate at least the effects of the dominant atmospheric phenomena. The fact that the description of some physical processes is only approximate and the fact that numerical models simulate only processes with certain spatial and temporal scales induce forecast errors. (Computer resources availability is one of the main factors that limits the complexity and the resolution of numerical models and data assimilation procedures, since, to be useful, numerical predictions must be produced in a reasonable amount of time.)

These two sources of forecast errors cause weather forecasts to deteriorate with forecast time. A third source of forecast error that is less important in the short and medium forecast range (say up to 10 days) but that can be very important for longer forecast ranges is related to the system boundary conditions (e.g. soil moisture content, ice coverage, vegetation).

It is worth pointing out that the system initial conditions will always be known only approximately, since each piece of data is characterized by an error that depends on the instrumental accuracy. In other words, small uncertainties related to the characteristics of the atmospheric observing system will always characterize the initial conditions. The growth of small initial errors into large forecast errors is due to the chaotic behavior of the atmosphere that implies that two initial states only slightly differing would depart one from the other very rapidly as time progresses.

A dynamical system shows a chaotic behavior if orbits exhibit sensitive dependence to initial conditions. An orbit is characterized by sensitive dependence if most other orbits that pass close to it at some point do not remain close to it as time advances. The atmosphere exhibits this behavior.

The top-left panel of Fig. 3 shows a very intense storm that crossed France and Germany during the 26^th of December 1999, and the other three panels of Fig. 3 show three 2-day forecasts started from very similar initial conditions at 12UTC (corresponding universal time) on 24 December 1999. The differences among the three initial conditions were comparable to estimated analysis errors. After only 2 days of numerical integration, the three forecasts evolved into very different atmospheric situations. In particular, note the different positions of the cyclone forecast over western Germany. The first forecast (Fig. 3, top-right) wrongly positioned the cyclone over Ireland, the second forecast (Fig. 3, bottom-left) correctly positioned the cyclone over Germany and the third forecast (Fig. 3, bottom-right) moved the cyclone too quickly over the Baltic Sea. This is an example of orbits initially close together and then rapidly diverging during the time evolution.

Another example of sensitivity to the initial state is shown in Fig. 4. Figure 4 shows the forecasts for air temperature in London given by 33 different forecasts started from very similar initial conditions for two different dates, the 26^th of June of 1995 and the 26^th of June 1994. There is a clear different degree of divergence among the 33 forecasts during the two cases. All forecasts stay close together up to forecast day 10 for the first case (Fig. 4, top panel), while they all diverge already at forecast day 3 in the second case (Fig. 4, bottom panel). The level of spread among the different forecasts can be used as a measure of the predictability of the two atmospheric states.

A complete description of the weather prediction problem can be stated in terms of the time evolution of an appropriate probability density function (PDF) in the atmosphere's phase space (Fig. 5). The predicted PDF of possible future atmospheric states is a function greater than zero in the phase space regions where the atmospheric state can be, with maximum values identifying the most probable future states.

The problem of the prediction of the PDF can be formulated exactly through the continuity equation for probability (Liouville equation). Unfortunately, with the current computer power availability the Liouville equation that describes the PDF time evolution can only be solved for simple systems characterized by a limited number of degrees of freedom. Furthermore, the initial PDF is not well known. Ensemble prediction based on a finite number of deterministic integrations is a feasible method to predict the PDF beyond the range of linear error growth. One of the by-products of ensemble prediction is the possibility to estimate the forecast skill of a deterministic forecast, or, in other words, to forecast the forecast skill.

Ensemble prediction systems should be designed to simulate all sources of forecast errors, in particular errors due to initial and model uncertainties. The relative importance of these two sources of forecast errors depends on the characteristics (e.g. spatial and temporal scales) of the phenomena under investigation. For large-scale atmospheric patterns, research studies performed with state-of-the-art numerical models have indicated that for short and medium time ranges (say for forecasts up to 3 to 5 days) errors are mainly due to initial uncertainties. Model errors due to the parameterized physical processes start having a non-negligible effect after forecast day 3. By contrast, for the prediction of small scale low-pressure systems and associated precipitation fields, for example, model errors can be as important as initial uncertainties at forecast day 2 or even earlier. For forecast times longer than 10 days (monthly and seasonal prediction) other error sources should be simulated. For examples, the possible influence of uncertainties in the boundary conditions (e.g. in the soil moisture content, in the ice and vegetation coverage) should be taken into account.

At the time of writing (August 2000), medium-range (i.e. for forecasts up to 15 days) ensemble systems are part of the routine operational suites at the Canadian Meteorological Center (CMC, Montreal, Canada, http://www.cmc.ec.gc.ca), at the European Centre for Medium-Range Weather Forecasts (ECMWF, Reading, UK, http://www.ecmwf.int) and at the National Centers for Environmental Predictions (NCEP, Washington, US, http://www.ncep.noaa.gov). These three Meteorological Centres run daily ensemble systems and can deliver to their users/customers probabilistic forecasts. Beside this operational activity, many Universities and other national and regional meteorological centers have been involved in research and experimental activities in this field.

The three ensemble systems operational at CMC, ECMWF and NCEP are all based on a finite number of numerical integrations starting from perturbed initial conditions, but they differ in the way the perturbed initial conditions are constructed. The ensemble systems operational at CMC and ECMWF include different schemes to simulate model errors while the NCEP system does not simulate them. Furthermore, they are different in their ensemble size, resolution and forecast-length of the numerical integration (Table 1). (It should be stressed that computer power availability is the main factor affecting configuration parameters as ensemble size and resolution.)

Despite their differences, schematically each member e_j (with j=1,N_ens) of any of these operational ensembles is defined by the time integration

(Eq. 5)

of the perturbed model equations

(Eq. 6)

starting from perturbed initial conditions e_j(t₀). Note that in Equation [6] the term that identifies the contribution to the full equation tendency of the parameterized physical processes, P_j(y,t), is different for each ensemble member. This represents the fact that model errors due to parameterized physical processes are simulated in the ensemble system.

The perturbed initial conditions e_j(t₀) are generated to represent the initial uncertainties. This can be accomplished by following different approaches with the constraint that the number of initial perturbations is limited to few tens. In probabilistic terms, this is equivalent to say that the initial time PDF can only be sampled a small number of times.

At ECMWF, the perturbed initial conditions are defined by adding to the best estimate of the initial state e₀(t₀) (computed by minimizing J(..,..), see Equation [2]), an ensemble of initial perturbations de_j(t₀)

(Eq. 7)

The initial perturbations de_j(t₀) have been designed to sample the components of the initial uncertainties with a maximum growth during the forecast time, i.e. for times t>t₀ and during the data assimilation time, i.e. for times t<t₀. This choice is based on the hypothesis that the components of the initial uncertainties with the fastest growth during the system time evolution and during the data-assimilation time period have the dominant effect on the forecast accuracy. Practically, these ‘growing directions’, called singular vectors, are computed by applying complex techniques based on iterative integrations of forward and adjoint tangent versions of the non-linear model.

At NCEP, as at ECMWF, initial perturbations are added to the best estimate of the initial state e₀(t₀). But compared to ECMWF, the NCEP initial perturbations are designed to sample only the components of the initial uncertainties that grow during the data-assimilation time interval, i.e. for times t<t₀. As a consequence, the two sets of initial perturbations can differ substantially. For example, ECMWF initial perturbations are characterized by smaller scales and by faster growth during the first 2 days of forecast integration than the NCEP perturbations.

At CMC, a different approach is followed. First, given the set of observations o(t), an ensemble of perturbed observations o_j(t) of is generated, o_j(t)=o(t)+do_j. Then, the ensemble of N_ens initial perturbations is generated by performing N_ens data assimilation cycles (i.e. by minimizing Equation [3] j-times, each time using a different set of perturbed observations o_j(t)), so that each perturbed initial state e_j(t₀) is the best estimate of the atmospheric state given by the set of perturbed observations o_j(t). It should also be mentioned that, since the CMC ensemble simulated also model errors, each data assimilation procedure is performed using a different model version, see next section.

The differences between the three methods seem subtle, but they have a significant impact on the characteristics of the ensemble systems (e.g. on the ensemble dispersion). The debate on what is the best approach is still open.

As it is the case for the simulation of initial uncertainties, there is not yet agreement within the scientific community of which is the best way to simulate model errors.

At CMC, different and on average equally skilful parameterization-schemes are used when numerically integrating each perturbed member. The rationale of this approach is that despite the fact that these different schemes perform equally on average, they can perform significantly differently on single occasions. Schematically, the CMC approach can be described as follows. Suppose that the following different parameterization schemes are given: three schemes that simulate moist processes (C₁, C₂ and C₃), three schemes that simulate turbulent diffusion processes (D₁, D₂ and D₃) and two radiation schemes (R₁ and R₂). Using a different combination of the schemes an ensemble system with 18 perturbed members, each integrated with a different model, can be designed. Schematically, each member of the CMC ensemble is defined by Equation [5] with

(Eq. 8)

where the full tendency due to the parameterized physical processes is computed by adding the tendencies computed by different combinations of the schemes simulating moist processes, turbulent diffusion and radiation (C_k,j means that the j-th ensemble member is integrated using the C_k).

At ECMWF, the random component of model errors due to parameterized physical processes is simulated by stochastically perturbing the tendency due to the physical processes. Schematically, each member of the ECMWF ensemble is defined by Equation [5] with

(Eq. 9)

where P is the unperturbed tendency due to all the parameterized physical processes, and r_j is a vector of random numbers. At the time of writing (September 2000), in the ECMWF operational ensemble system the random numbers r_j are uniformly sampled in the interval –0.5<r_j<0.5.

Ensemble prediction products are becoming increasingly popular. Figures 6 and 7 show two examples of weather products designed, respectively, for a forecaster interested only in a single location and for a forecaster interested in a large area. Figure 6 shows the 10-day ensemble prediction of temperature, geopotential height, precipitation and cloud cover for London given by the ECMWF ensemble prediction system started on the 28^th of March 2000. Figure 6 shows, for example, that winds speed will decrease during the first 3 forecast days, that temperature will rise slightly, and that there could be intense precipitation (up to 20mm/12h) on Sunday the 2^nd of April. The fact that the size of the box-and-whiskers increases during the forecast time especially after Saturday the 1^st of April indicates that the ensemble spread increases, thus suggesting future less predictable situations.

The second product has been designed for a forecaster interested in assessing the accuracy of an 84h prediction of mean-sea-level-pressure (MSLP) and precipitation over Northeastern Spain. The top-left panel of Fig. 7 shows the observed MSLP (more precisely the ECMWF analysis) and the 24h precipitation forecast given by the ECMWF high-resolution model that can be considered as a good approximation of the large scale feature of the actual precipitation field. This panel shows that more than 40mm of rain affected a large area across the Spanish-French border. The top-right panel shows the 84h forecast given by the ECMWF unperturbed (control) forecast. The bottom-left panel shows the 84h MSLP forecast given by the ECMWF ensemble-mean and the ensemble standard deviation (shading). The bottom-right panel shows the 84h probability forecast of more than 20mm/day of precipitation given the ECMWF ensemble system. The two bottom panels can be used to assess the possible accuracy of the forecast given by the control forecast (top-right panel). The bottom-left panel shows that the ensemble-mean forecast predict an atmospheric flow (at the surface) similar to the one predicted by the control forecast, and that the ensemble standard deviation, which is a measure of the agreement/disagreement among the ensemble members, is not particularly large in the area of interest (i.e. across the Spanish-French border). This can be used as an indication that the ensemble spread in the are of interest is rather small and thus that the situation should be predictable. The bottom-right panel shows that there is a probability of more than 40% that intense precipitation will affect the are of interest, again confirming that there is good agreement among the ensemble members in predicting this phenomenon.

These two figures constitute simple examples of how ensemble prediction can be used to complement single deterministic forecasts with probabilistic information to try to assess the forecast accuracy. It is worth to mention that ensemble products have started been used experimentally in business applications such as ship-routing and energy-demand predictions.

The primary purpose of ensemble systems is to estimate the probability density function (PDF) of forecast states. As a consequence, the quantitative evaluation of ensemble systems should be based on the comparison of the forecast PDF with the observed PDF. In particular, verification measures should be designed to assess the statistical consistency and usefulness of the predicted PDF. These two properties should be assessed by considering the first-order moment of the predicted PDF (the ensemble-mean) and the second-order moment (ensemble standard deviation). Verifications should be performed both a grid-point level (i.e. for single locations) and considering large-scale atmospheric patterns (i.e. regimes characterized by a blocked or a zonal flow over the Euro-Atlantic sector). The skill of an ensemble system should be compared with the skill of some reference systems (benchmarks) such as climatology or simpler PDF-prediction systems.

Numerical weather forecasts are often used by decision-makers to decide whether or not to take an action to protect against a possible loss. Typically, the decision-maker would spend an amount C if an event is predicted to protect himself against a loss L (with L>C). The economic value of a forecasting system can be assessed by using skill measured defined by coupling contingency tables and cost-loss decision. The economic value of a forecast can be defined as a function of the false alarm rate and the probability of detection of the system. Since this measure is defined for both categorical and probabilistic forecast, it can be used to compare the economic value of a single forecast and of an ensemble forecasting system.

Figure 8 shows, for December 1999 over Europe, the average economic value of the t+120h prediction for two events, 24-h accumulated precipitation greater than 5 and 10 mm/d, for three different forecasts, specifically the single deterministic forecasts given by the ECMWF unperturbed (control) forecast and by the ensemble-mean, and the probabilistic prediction generated using the whole ensemble system (the economic value varies from 0, for a forecast as skilful as climatology, to 1 for a perfect forecast).

Figure 8 shows that the economic value for he ensemble probabilistic prediction is definitely larger than the economic value of the two probabilistic predictions, especially for small cost/loss ratios. Considering the deterministic predictions, Fig. 8 shows that (for the period and the event considered) the ensemble-mean and the control forecast have about the same economic value for the 5mm/d threshold (Fig. 8 top), while for the 10mm/d threshold (Fig. 8 bottom) the control forecast performs better for small cost/loss ratios but worse for large values.

This result indicates that for December 1999 a decision-maker interested in predicting a binary event "rainfall greater than 5 or 10 mm/d" over Europe would have had a higher return if he took his decisions (protect/non-protect) according to the ensemble forecast than to any single deterministic forecast.

The operational implementation of ensemble prediction systems has changed the approach to numerical weather prediction from deterministic (i.e. based on a single forecast) to probabilistic. Ensemble systems provide a possible way to estimate the probability distribution function of forecast states. They have been developed following the notion that uncertainties in the initial conditions and in the model formulation are the main sources of forecast errors.

Ensemble prediction systems are particularly useful, if not necessary, to provide early warnings of the risk of extreme weather events. For example, ensemble systems can be used to predict probabilities of intense precipitation events. The economic value of ensemble prediction is higher than the economic value of single deterministic forecasts. Global ensemble systems can be used to provide boundary and initial conditions for higher-resolution, limited area ensemble prediction systems.

Research activity is in progress in many different areas to further improve the accuracy of the operational ensemble prediction system. Work is in progress to improve the representation of initial and model errors, and furthermore to possibly use ensemble approaches to data assimilation. Multi-model, multi-analysis ensemble systems based on a set of integrations performed with different models and starting from analyses constructed using different data assimilation schemes are under investigation. Work is also in progress to further develop ensemble products that can be used more easily in businesses heavily affected by weather (shipping industry, energy sector, safety and protection agencies).

Haltiner, G. J., & Williams, R. T. (1979) Numerical Prediction and Dynamic Meteorology. John Wiley & Sons, pp. 477.

Holton, J. R. (1982) An Introduction to Dynamic Meteorology. Academic Press Inc., pp. 511.

Lorenz, E. N. (1993) The essence of Chaos. University College London (UCL) Press, pp. 227.

Richardson, L. F. (1922) Weather Prediction by Numerical Process. Cambridge University Press (reprt. Dover, New York).

Wilks, D. S. (1995), Statistical Methods in the Atmospheric Sciences. Academic Press, pp. 467.

ECMWF Workshop Proceedings (1992), New Developments in Predictability, 13-15 November 1991. ECMWF, Shinfield Park, Reading RG2 9AX, UK, pp. 334.

ECMWF Workshop Proceedings (1999), Workshop on Predictability, 20-22 October 1997. ECMWF, Shinfield Park, Reading RG2 9AX, UK, pp. 372.

(back to top of the page)