Weather observations have always been a key ingredient in weather prediction. They are used to help determine the starting point for any forecast. In traditional numerical weather prediction, they are combined with a previous short-term forecast to produce a complete initial state for the whole of the atmosphere and related Earth system components. This is done in a complex process called data assimilation, and much research into weather prediction is dedicated to improving this data assimilation step. Recently, there have been a number of developments performing weather forecasting through machine learning, for example as implemented in ECMWF’s Artificial Intelligence Forecasting System (AIFS), but these methods currently do not do away with data assimilation: they still require an optimal starting point for their forecasts, and this is provided by data assimilation. However, some of our staff have been working hard on a system, based on machine learning, that does not use data assimilation at all: neither in training nor in inference mode. Instead, forecasts are based on observations alone, without a separate step of establishing a complete initial state and without relying on reanalyses for the training.
Such an approach avoids two issues associated with data assimilation: the first is the detailed knowledge that is required of uncertainty in the observations and in the prior forecast state; and the second is the need to have a very accurate mapping between the observations, which may be radiation measured by satellites, and the model states – this is often not straightforward or even impossible. The forecasting method under development, called Artificial Intelligence–Direct Observation Prediction (AI–DOP), avoids these issues. It does so by operating directly on the physical quantities that are actually measured by meteorological observing systems. First results of this method are presented in this Newsletter. They show impressive performance for the first two days, although after that performance is currently not as good as that of other forecasting methods. This innovative approach is the most radical AI departure from physics-based forecasting methods we are investigating, and it will be interesting to see where it takes us.
The other feature articles in this Newsletter indicate the importance of computing at ECMWF. They cover plans for the modernisation of our Integrated Forecasting System (IFS); a review of 20 years of the Framework for Member State time-critical applications, which enables our Member and Co-operating State users to run time-critical work on our high-performance computing facility with varying levels of monitoring and support; and an introduction to ECMWF Sites, a service that enables our staff and visiting scientists to create and publish websites.
This Newsletter also reviews our forecast performance in 2024. In addition to other highlights in our forecast skill, that article mentions the excellent results of the AIFS in experimental mode. The AIFS is due to be made operational later this year. This will be a fantastic operational milestone on our road to embracing machine learning in weather forecasting.
Florence Rabier
Director-General