A comparison of observing system experiments with the physics based IFS and the data driven AIFS

This study intercompares results from Observing System Experiments (OSEs) with the physics-based Integrated Forecast System (IFS) and the machine-learning based Artificial Intelligence Forecast System (AIFS), with a focus on the performance in the troposphere. The OSEs characterise the respective denial of all in-situ/conventional data, passive microwave (MW) radiances, infrared (IR) radiances, and Global Navigation Satellite Systems (GNSS) radio occultation (RO) data compared to a Control that uses the full observing system. The respective experiments use the same analyses for both forecasting systems, produced with the physics-based IFS.

Results highlight that both systems show significant forecast degradation when the selected observing systems are denied, with degradations over the troposphere that are mostly ball-park similar to degradations previously demonstrated in physics-based systems only. The denial of conventional data leads to the largest degradations over the Northern Hemisphere in both systems, and the denial of MW radiances is most detrimental over the Southern Hemisphere. However, the degradations in the AIFS tend to be larger than the ones found in the IFS, at least for the denial of the conventional data, MW or IR radiances. The largest increase in the degradation in the AIFS is found in the tropical troposphere, where impacts in terms of percentage increase of the forecast error can be doubled compared to the IFS. This is also the area for which the AIFS Control shows the largest benefits compared to the IFS Control. Some of the largest increases in impact in the AIFS are found for the conventional data, tentatively linked to larger changes in the distribution of the input data. The larger vulnerability to the loss of observations in the AIFS is, however, smaller than the benefit in forecast scores that the AIFS brings over the IFS for the levels and parameters considered. GNSS-RO data show a very strong forecast impact in the upper troposphere/lower stratosphere in the IFS, particularly in the tropics, and this is more muted in the AIFS, likely due to the focus on the troposphere during the training of AIFS.