| Home > Newsevents > Training > Rcourse_notes > DATA_ASSIMILATION > ASSIM_CONCEPTS > |
Data assimilation concepts and methods
March 1999
By F. Bouttier and P. Courtier
| Table of contents |
Table of contents >>
Next Section >>
1 . Basic concepts of data assimilation
Analysis. An analysis is the production of an accurate image of the true state of the atmosphere at a given time, represented in a model as a collection of numbers. An analysis can be useful in itself as a comprehensive and self-consistent diagnostic of the atmosphere. It can also be used as input data to another operation, notably as the initial state for a numerical weather forecast, or as a data retrieval to be used as a pseudo-observation. It can provide a reference against which to check the quality of observations.
The basic objective information that can be used to produce the analysis is a collection of observed values provided by observations of the true state. If the model state is overdetermined by the observations, then the analysis reduces to an interpolation problem. In most cases the analysis problem is under-determined1 because data is sparse and only indirectly related to the model variables. In order to make it a well-posed problem it is necessary to rely on some background information in the form of an a priori estimate of the model state. Physical constraints on the analysis problem can also help. The background information can be a climatology or a trivial state; it can also be generated from the output of a previous analysis, using some assumptions of consistency in time of the model state, like stationarity (hypothesis of persistence) or the evolution predicted by a forecast model. In a well-behaved system, one expects that this allows the information to be accumulated in time into the model state, and to propagate to all variables of the model. This is the concept of data assimilation.
Figure 1 . Representation of four basic strategies for data assimilation, as a function of time. The way the time distribution of observations ("obs") is processed to produce a time sequence of assimilated states (the lower curve in each panel) can be sequential and/or continuous.
Assimilation. Data assimilation is an analysis technique in which the observed information is accumulated into the model state by taking advantage of consistency constraints with laws of time evolution and physical properties.
There are two basic approaches to data assimilation: sequential assimilation, that only considers observation made in the past until the time of analysis, which is the case of real-time assimilation systems, and non-sequential, or retrospective assimilation, where observation from the future can be used, for instance in a reanalysis exercise. Another distinction can made between methods that are intermittent or continuous in time. In an intermittent method, observations can be processed in small batches, which is usually technically convenient. In a continuous method, observation batches over longer periods are considered, and the correction to the analysed state is smooth in time, which is physically more realistic. The four basic types of assimilation are depicted schematically in Fig. 1 . Compromises between these approaches are possible.
Figure 2 . A summarized history of the main data assimilation algorithms used in meteorology and oceanography, roughly classified according to their complexity (and cost) of implementation, and their applicability to real-time problems. Currently, the most commonly used for operational applications are OI, 3D-Var and 4D-Var.
Many assimilation techniques have been developed for meteorology and oceanography (Fig. 2 ). They differ in their numerical cost, their optimality, and in their suitability for real-time data assimilation. Most of them are explained in this volume.
ref: Daley 1991; Lorenc 1986; Ghil 1989
1.1 On the choice of model
The concepts developed here are illustrated by examples in the ECMWF global meteorological model, but they can be (and they have been) applied equally well to limited area models, mesoscale models, ocean circulation models, wave models, two-dimensional models of sea surface temperature or land surface properties, or one-dimensional vertical column models of the atmosphere for satellite data retrieval, for example. This presentation could be made in the general framework of an infinite-dimensional model (i.e. without discretization) with a continuous time dimension. This would involve some sophisticated mathematical tools. For the sake of simplicity, only the discrete, finite-dimensional problem will be addressed here.
In meteorology there are often several equivalent ways of representing the model state. The fields themselves can be represented as grid-point values (i.e. averages of the fields inside grid boxes), spectral components, EOF values, finite-element decomposition, for instance, which can be projections on different basis vectors of the same state. The wind can be represented as components
, vorticity and divergence 
, or streamfunction
and velocity potential 
, with a suitable
definition of the integration constants. The humidity can be represented
as specific or relative humidity or dew-point temperature, as long as temperature
is known. In the vertical, under the assumption of hydrostatic balance,
thicknesses or geopotential heights can be regarded as equivalent to the
knowledge of temperature and surface pressure. All these transforms do not
change the analysis problem, only its representation2. This may sound trivial, but it
is important to realize that the analysis can be carried out in a representation
that is not the same as the forecast model, as long as the transforms are
invertible. The practical problems of finding the analysis, e.g. the modelling
of error statistics, can be greatly simplified if the right representation
is chosen.Since the model has a lower resolution than reality, even the best possible analysis will never be completely realistic. In the presentation of analysis algorithms we will sometimes refer to the true state of the model. This is a phrase to refer to the best possible state represented by the model, which is what we are trying to approximate. Hence it is clear that, even if the observations do not have any instrumental error, and the analysis is equal to the true state, there will be some unavoidable discrepancies between the observed values and their equivalents in the analysis, because of representativeness errors. Although we will often treat these errors as a part of the observation errors in the mathematical equations below, one should keep in mind that they depend on the model discretization, not on instrumental problems.
1.2 Cressman analysis and related methods
One may like to design the analysis procedure as an algorithm in which the model state is set equal to the observed values in the vicinity of available observations, and to an arbitrary state (say, climatology or a previous forecast) otherwise. This formed the basis of the old Cressman analysis scheme (Fig. 3 ) which is still widely used for simple assimilation systems.
The model state is assumed to be univariate and represented as grid-point values. If we denote by
a previous estimate
of the model state (background) provided by climatology, persistence
or a previous forecast, and by 
, a set of 
observations of the same parameter,
a simple kind of Cressman analysis is provided by the model state 
defined at each grid point j according
to the following update equation:
|
where
is a measure of the distance between points 
and 
. 
is the background
state interpolated to point 
. The
weight function 
equals one if the grid point 
is collocated with observation 
. It is a decreasing function of distance which is zero if 
, where R is a user-defined constant (the
"influence radius") beyond which the observations have no weight.
Figure 3 . An example of Cressman analysis of a one-dimensional field. The background field
is represented as the blue function, and the observations in
green. The analysis (black curve) is produced by interpolating between the
background (grey curve) and the observed value, in the vicinity of each
observation; the closer the observation, the larger its weight.There are many variants of the Cressman method. One can redefine the weight function, e.g. as
. A more general
algorithm is the successive correction method (SCM)3. One of its features is that
the weights can be less than one for 
, which
means that a weighted average between the background and the observation
is performed. Another one is that the updates can be performed several times,
either as several iterations at a single time in order to enhance the smoothness
of corrections, or as several corrections distributed in time. With enough
sophistication the successive correction method can be as good as any other
assimilation method, however there is no direct method for specifying the
optimal weights.ref: Daley 1991
1.3 The need for a statistical approach
The Cressman method is not satisfactory in practice for the following reasons:
| • if we have a preliminary estimate of the analysis with a good quality, we do not want to replace it by values provided from poor quality observations. |
• when going away from
an observation, it is not clear how to relax the analysis toward the
arbitrary state, i.e. how to decide on the shape of the function  . |
| • an analysis should respect some basic known properties of the true system, like smoothness of the fields, or relationship between the variables (e.g. hydrostatic balance, or saturation constraints). This is not guaranteed by the Cressman method: random observation errors could generate unphysical features in the analysis. |
Because of its simplicity, the Cressman method can be a useful starting tool. But it is impossible to get rid of the above problems and to produce a good-quality analysis without a better method. The ingredients of a good analysis are actually well known by anyone who has experience with manual analysis:
| 1) one should start from a good-quality first guess, i.e. a previous analysis or forecast that gives an overview of the situation, |
| 2) if observations are dense, then one assumes that the truth probably lies near their average. One must make a compromise between the first guess and the observed values. The analysis should be closest to the data we trust most, whereas suspicious data will be given little weight. |
| 3) the analysis should be smooth, because we know that the true field is. When going away from an observation, the analysis will relax smoothly to the first guess on scales known to be typical of the usual physical phenomena. |
| 4) the analysis should also try to respect the known physical features of the system. Of course, it is possible in exceptional cases that unusual scales and imbalances happen, and a good analyst must be able to recognize this, because exceptional cases are usually important too. |
Loosely speaking, the data that can go into the analysis system comprises the observations, the first guess and the known physical properties of the system. One sees that the most important feature to represent in the analysis system is the fact that all pieces of data are important sources of information, but at the same time we do not trust any of them completely, so we must make compromises when necessary. There are errors in the model and in the observations, so we can never be sure which one to trust. However we can look for a strategy that minimizes on average the difference between the analysis and the truth.
To design an algorithm that does this automatically, it is necessary to represent mathematically the uncertainty of the data. This uncertainty can be measured by calibrating (or by assuming) their error statistics, and modelled using probabilistic concepts. Then the analysis algorithm can be designed on a formal requirement that in the average the analysis errors must be minimal in a sense that is meaningful to the user. This will allow us to write the analysis as an optimization problem.
ref: Lorenc 1986
Training Course Notes Front Page >>
Table of contents >>
Next Section >>
1 although it can be overdetermined locally in data-dense areas
2 At ECMWF, the analysis problem is currently formulated in terms of the spectral components of vorticity, divergence, temperature, grid-point values of specific humidity, on surfaces defined by the hybrid coordinate, and logarithm of surface pressure, just like in the forecast model. In winter 1998 the model state dimension was about 6.106.
3 In the recent literature this name is often replaced by observation nudging which is more or less the same thing. The model nudging is a model forcing technique in which the model state is relaxed toward another predefined state.
 |
06.06.2002 |
 |
© ECMWF |
