IFS upgrade provides more skilful ensemble forecasts

Mark Rodwell, Michail Diamantakis, Peter Düben, Martin Janoušek, Simon Lang, Inna Polichtchouk, Fernando Prates, Chris Roberts, Filip Váňa


On 11 May 2021, ECMWF implemented a substantial upgrade of its Integrated Forecasting System (IFS). As with almost all upgrades, this involved contributions from many teams within the Centre. IFS Cycle 47r2 includes changes to the forecast model, but not to the data assimilation system. The upgrade is neutral for the medium-range deterministic high-resolution (HRES) forecast but brings benefits to the medium- and extended-range ensemble forecasts (ENS). Cycle 47r2 is the culmination of two strands of work:

  •  A change from double precision to single precision in HRES and ENS forecasts​
  •  An increase in the number of model levels from 91 to 137 in ENS forecasts

Forecast model

Previous versions of the IFS have used ‘double precision’, where each number is stored using 64 bits of memory. This is often more accurate than required when we consider observational errors and model approximations. Single precision, in which each number is stored with 32 bits of memory, offers the prospect of freeing up memory and, importantly, increasing processing speeds. Single precision of the IFS started as a research project in collaboration with the University of Oxford and as part of the OpenIFS effort. Similar lines of research were pursued in the COSMO (Consortium for Small-scale Modelling) model. It then became a collaborative project across many ECMWF teams. With more people working on the project, forecast skill became increasingly neutral over time with respect to double precision, up to a point where it could be incorporated into our operational forecasts. This allows computational savings to be made which can be used to achieve skill improvements. Figure 1 shows the computational changes to the ensemble forecast. Faster core processing (green circles) of single-precision data permits a 50% increase in ENS model levels from 91 to 137. Even with this increase in levels, data transferred (red arrows) between the memory on each node (yellow boxes) is reduced because it is now in single precision.

FIGURE 1 The computational change from (a) IFS Cycle 47r1 with double precision (DP) and 91 levels in the ensemble forecast to (b) IFS Cycle 47r2 with single precision (SP) and 137 levels in the ensemble forecast allows faster core processing (green circles) and reduced data transfer (red arrows) between the memory on each node (yellow boxes).

Double precision is still used throughout the data assimilation process, and some calculations in the forecast do still require double precision. The most expensive of those, such as the calculation of the associated Legendre polynomials and the finite-element integral operators of the vertical discretisation, are only done once and are not repeated during time-stepping. Hence, there is minimal impact on computational efficiency. Further detailed experimentation helped us to identify a few other calculations in parts of dynamics and physics and the stochastic physics perturbations that need to be secured with double precision. However, those also represent a very small part of the total computational load. Note also that GRIB encoding is unchanged, so archived files remain the same size.

The change to 137 levels brings us one step closer to a more seamless ensemble data assimilation and forecasting system. The need for vertical interpolation when generating ensemble initial conditions is now greatly reduced as the ensemble of data assimilations (EDA) is already run with 137 levels. The consistency with the HRES vertical resolution should also aid the evaluation process of future cycles. Technical changes to the ensemble include the calculation of singular vector perturbations with 137 levels.

FIGURE 2 HRES scorecard of IFS Cycle 47r2 versus IFS Cycle 47r1, verified by the respective analyses and observations at 00 and 12 UTC, based on 619 forecast runs in the period May 2020 to April 2021. The HRES is run at TCo1279 resolution, corresponding to a horizontal grid spacing of about 9 km.

Impact on medium- and extended-range forecasts

The goal for the implementation of single precision was neutrality in HRES scores, together with major computational cost savings. Neutrality would be demonstrated in an HRES scorecard (Figure 2) with approximately a third of the boxes being grey, a third red and a third blue, and with little more than 5% of the red and blue boxes being statistically significant at the 5% significance level (indicated by triangles). As can be seen in Figure 2, this has largely been achieved. A possible exception is a degradation (typically less than 1%) in stratospheric extratropical geopotential height scores.

The neutrality for the HRES is illustrated in Figure 3 by track forecasts of Hurricane Laura. While agreement cannot be perfect for a chaotic system, the medium-range track differences between single and double precision are much smaller than the spread of the ensemble, which represents the impacts of initial and model uncertainty. More generally, the impact of single precision on HRES tropical cyclone track and intensity scores is neutral.

FIGURE 3 Eight-day tracks of Hurricane Laura from 12 UTC on 22 August 2020 in high-resolution deterministic forecasts with double precision (red) and single precision (blue) along with those from the operational ensemble at the time (grey).

The increase in vertical resolution from 91 to 137 levels has been introduced to all ENS forecasts in the medium to the extended range. The ENS scorecard is shown in Figure 4. The change leads to statistically significant improvements to many ENS scores of about 0.5–2% throughout most of the free atmosphere. Stratospheric temperature scores are greatly improved, typically by 5–20%. This is, among other things, due to a weaker growth of temperature biases because the ENS can better resolve gravity waves in the vertical. Figure 5 shows this improvement at day 10, but it persists into the extended range. The mean cooling difference below 600 hPa (Figure 5 bottom panel) acts to decrease the warm bias around 850 hPa. It improves tropical medium-range scores at that level by over 6%. It does also slightly increase the tropical near-surface cool bias, and this is reflected in the 2‑metre temperature scores in Figure 4, which are degraded by up to 1% by day 14. Ten-metre wind scores are also slightly degraded by 0.1–0.3%.

The extra levels mean that sharper inversions can be resolved. For example, the ensemble vertical profile product now uses 34 model levels below 700 hPa instead of the previous 22. The Cycle 47r2 test profile in Figure 6, which uses the new mapping of model levels, shows a slightly sharper thermal inversion at around 850 hPa than the Cycle 47r1 profile. Users will need to ensure that they extract the correct model levels when creating their own forecast products.

FIGURE 4 ENS scorecard of IFS Cycle 47r2 versus IFS Cycle 47r1 for medium-range forecasts up to forecast day 15, verified by the respective analyses and observations at 00 UTC based on 408 ENS forecast runs in the period November 2019 to April 2021. The ENS is run at TCo639, corresponding to a horizontal grid spacing of about 18 km.

Tropical cyclones show reduced intensity errors (see Figure 7a). This is largely associated with reduced bias. There is a mean reduction of about 2 hPa in central pressure in the medium range, increased spread, and improved reliability as measured by the spread-error agreement. The cycle is neutral in terms of track errors (Figure 7b). Along with the increased tropical cyclone intensity, other tropical activity is increased: calculating anomalies from the operational extended-range re‑forecasts may be advisable.

A key source of sub-seasonal predictability is the Madden–Julian Oscillation (MJO). Out to the extended range, the amplitude of the MJO is better sustained: the amplitude loss by day 15 is now about 15% rather than the previous value of about 20% (Figure 8). There is also an increase in MJO spread, improved reliability and better scores. Changes come mostly from improvements in tropical zonal winds at 200 hPa.

FIGURE 5 Zonal means of mean temperature errors at a lead time of 10 days in the ensemble control forecast for (a) IFS Cycle 47r1, (b) IFS Cycle 47r2, and (c) the difference between IFS Cycles 47r2 and 47r1. More saturated colours indicate statistical significance at the 5% level using a t-test accounting for temporal correlation. Evaluated over all forecasts between 25 November 2019 and 28 February 2020 as well as 10 May 2020 and 7 November 2020.

Forecast outputs

More frequent tropical cyclone track updates are now available with the inclusion of forecasts from 6 and 18 UTC initial times, alongside those of the 0 and 12 UTC forecasts. More snowfall Extreme Forecast Index (EFI) and Shift of Tails (SOT) products are now available with the inclusion of 3-, 5-, 10- and 15‑day accumulation periods, in addition to the previous 1‑day accumulations. A selection of new specialist climatological model parameters includes some which describe the characteristics of topographic features smaller than the model grid box, some which are used within radiation calculations, and the ‘Logarithm of surface roughness length for heat’.

FIGURE 6 The vertical structure of temperature (red) and moisture (dewpoint, green) in tephigram format in (a) IFS Cycle 47r1 in a 60-hour forecast from 21 January 2021 00 UTC at 20.03°S 90°W and (b) the same forecast in IFS Cycle 47r2. Shaded bands denote the minimum, 25th and 75th percentiles and maximum for temperature and dewpoint ENS distributions at each level. The median dewpoint value is shown by the solid line within the dark green shading. The other solid line represents HRES and a thick dashed line represents the Control forecast.

FIGURE 7 Root-mean-square errors (RMSE) in the ensemble mean of (a) tropical cyclone (TC) intensities along with the standard deviation (spread) among ensemble members and (b) tropical cyclone locations along with the standard deviation (spread) among ensemble members. Results are based on all TC basins for the periods 25 November 2019 to 28 February 2020 and 10 May 2020 to 30 November 2020. The numbers at the top of the panels indicate the number of TCs which could be evaluated at each lead time. The bars indicate 95% confidence intervals.

FIGURE 8 Score differences (Cycle 47r2 with 137 levels minus Cycle 47r2 with 91 levels) for the bivariate real-time multivariate Madden–Julian Oscillation (RMM) index based on re-forecasts initialised at the start of each month for the period 1989–2016. The differences shown are for (i) the fair version of the continuous rank probability skill score (FCRPSS), (ii) the bivariate anomaly correlation, (iii) the bivariate root-mean-square error (RMSE), (iv) the bivariate spread of the ensemble with respect to ensemble mean, and (v) the amplitude of the RMM index. Bivariate scores are calculated and verified against the RMM index constructed from the ERA5 reanalysis. Triangles indicate increased (pointing up) and decreased (pointing down) values, which are significant at the 1% level when the shading is more saturated.


The change to single precision in forecast mode for the HRES and ENS systems has freed up computing resources to be used to enhance forecast skill. In IFS Cycle 47r2, the choice has been made to use these resources to make the model levels used in the ENS match those of the EDA and HRES systems. This represents an important step within ECMWF’s ten‐year Strategy 2021–2030, which highlights “work towards a seamless integration from the ensemble of data assimilations to the ensemble forecast system”. The fact that the ENS and HRES now have the same model levels should also facilitate future cycle development. The new cycle increases ENS forecast skill by typically 0.5–2% in the free atmosphere, but by 5–20% for stratospheric temperatures at 50 hPa and by 6% in the tropical troposphere. It also intensifies tropical cyclones, thus reducing intensity errors and improving reliability, and it helps to better sustain the amplitude of the Madden–Julian Oscillation into the extended range.


Further reading

Lang, S.T.K., A. Dawson, M. Diamantakis, P. Dueben, S. Hatfield, M. Leutbecher et al.: More Accuracy with Less Precision, submitted to QJRMS.

Polichtchouk, I., T. Stockdale, P. Bechtold, M. Diamantakis, S. Malardel, I. Sandu et al.: 2019. Control on stratospheric temperature in IFS: resolution and vertical advection. ECMWF Technical Memorandum No. 847.

Váňa, F., P. Düben, S. Lang, T. Palmer, M. Leutbecher, D. Salmond et al.: 2017. Single Precision in Weather Forecasting Models: An Evaluation with the IFS. Monthly Weather Review, 145(2), 495–502.​