|
|
Training
Course Notes Front Page >>
Table of contents >>
Next Section >>
Previous Section >>
5 . A simple scalar illustration of
least-squares estimation
Let us assume that we need to estimate the temperature
of a room.
We have a thermometer of known accuracy (the standard deviation of measurement error) and we observe
, which is considered to have expectation (i.e. we assume that the observation is unbiased)
and variance . In the absence of any other information the
best estimate we can provide of the temperature is , with accuracy .
However we may have some additional information about the
temperature of the room. We may have a reading from another, independent
thermometer, perhaps with a different accuracy. We may notice that everyone
in the room is wearing a jumper-another timely piece of information from
which we can derive an estimate, although with a rather large associated
error. We may have an accurate observation from an earlier date, which can
be treated as an estimate for the current time, with an error suitably inflated
to account for the separation in time. Any of these observations could be
treated as a priori or background information, to be
used with in estimating
the room temperature. Let our background estimate be , of expectation (i.e.
it is unbiased) and of accuracy . Intuitively
and can be
combined to provide a better estimate (or analysis) of than any of these taken alone. We are going
to look for a linear weighted average of the form:
| |
|
|
which can be rewritten as , i.e. we look for a correction to the background
which is a linear function of the difference between the observation and
the background.
The error variance of the estimate is:
where we have assumed that the observation and background errors are uncorrelated.
We choose the optimal value of that minimizes
the analysis error variance:
which is equivalent to minimizing (Fig. 5 )
Figure 5 . Schematic representation of the variational
form of the least-squares analysis, in a scalar system where the observation
is in the same space as the model
: the cost-function terms and are both convex and tend to "pull" the
analysis towards the background and the
observation , respectively. The minimum of
their sum is somewhere between and , and is the optimal least-squares analysis.
| |
• In the limiting case
of a very low quality measurement ( ), and the analysis remains equal to the background. |
| |
• On the other hand, if
the observation has a very high quality ( ), and the analysis is equal to the observation. |
| |
• If both have the same
accuracy, , and the analysis
is simply the average of and , which reflects the fact that we trust as much the
observation as the background, so we make a compromise. |
| |
• In all cases, , which means that the analysis is a weighted average of the background
and the observation. |
These situations are sketched in Fig. 6 .
Figure 6 . Schematic representation of the variations
of the estimation error , and
of the optimal weight that determines
the analysis , for
various relative amplitudes of the background and observation standard errors
( ).
It is interesting to look at the variance of analysis error
for the optimal :
or
which shows that the analysis error variance is always smaller than both
the background and observation error variances, and it is smallest if both
are equal, in which case the analysis error variance is half the background
error variance.
Training Course Notes Front Page >>
Table of contents >>
Next Section >>
Previous Section >>
|