Angela Iza Wong, Ecuador Hydrometeorological Service and WMO Fellow at ECMWF
Rainfall in Ecuador is anything but predictable. The country’s diverse geography makes forecasting a real challenge.
Over the past year, I’ve had the opportunity to explore this complexity through a World Meteorological Organization (WMO) fellowship at ECMWF, where I worked with models to evaluate how well they capture Ecuador’s rainfall patterns. What I found was an interesting combination of strengths, limitations, and promising innovations that could shape the future of weather forecasting in tropical regions.
The project started with a rigorous quality control process for rainfall data from weather stations across Ecuador, aiming to evaluate reanalysis (ERA5) and satellite products. In the next phase, I analysed ECMWF's models, including the Integrated Forecasting System (IFS), several versions of the Artificial Intelligence Forecasting System (AIFS), other AI models including GraphCast by Google DeepMind, and the high-resolution Destination Earth model (DestinE). Each model was assessed for its ability to detect both everyday and extreme rainfall events across Ecuador’s diverse landscapes and elevations.
Rainfall observations across Ecuador and quality control procedures
The initial phase of my project involved compiling and processing rainfall data from both conventional (i.e., traditional ground-based stations that use manual instruments and human oversight) and automated weather stations across Ecuador.
To ensure the data was reliable, it underwent a thorough quality control procedure, based on standards set by WMO. Out of the data from 821 conventional stations collected between 1980 and 2024, 216 stations met the quality criteria — defined as having at least 50% data availability — and were retained for further analysis.
This dataset, spanning the period from 1980 to 2020, was used to evaluate ERA5 reanalysis and satellite-derived precipitation estimates. Quality-controlled data from 30 conventional stations for 2023 to 2024 were also used to validate high-resolution precipitation forecasts generated by physical models (IFS), artificial intelligence models (AIFS and GraphCast), and Destination Earth (DestinE) models across multiple varied regions of Ecuador.
Figure 1: Observation available in Ecuador in 2023 by region. The model orography is derived from the High-Resolution Model (9 km) of ECMWF.
How well do satellites see the rain?
In the second phase, my study evaluated how well historical and satellite-based rainfall estimates matched up with real-world observations. This involved comparing ERA5 reanalysis data and satellite products — Integrated Multi-satellite Retrievals for GPM (IMERG) and Multi-Source Weighted-Ensemble Precipitation (MSWEP) — against a carefully curated dataset from 216 weather stations across Ecuador, covering the years 1980 to 2020. The goal was to identify biases, understand how well these models detected extreme events, and assess the influence of topography on precipitation.
A case study examined the 1998 El Niño event, which resulted in heavy rainfall and flooding in Ecuador. My results show that IMERG provides the most accurate estimates in lowland regions (at or below 800 meters in elevation); however, accuracy decreases in mountainous areas above 2,400 meters. ERA5 consistently detects but overestimates heavy rainfall along the coast. These findings are detailed in a scientific manuscript, "Evaluation of Precipitation Observations Across Ecuador", submitted to Atmospheric Science Letters in June 2025 (manuscript currently under revision).
Figure 2: (a) Annual moving average of the daily bias for ERA5, IMERG, and MSWEP (1980–2020). Data after 2015 are indicated by dashed lines, with El Niño years highlighted in red and La Niña years in blue. (b) Box plot of bias for stations at different altitudes.
How do traditional and AI models compare?
The third, and final, phase focused on evaluating both high-resolution physical models (IFS) and artificial intelligence-driven forecasting systems (AIFS). Several AIFS experiments were assessed, including models trained on IMERG and MSWEP data, and compared to GraphCast and the high-resolution DestinE model. To assess the performance of these models across various regions, I used quality-controlled data from 30 conventional stations, collected between 2023 and 2024.
A comprehensive evaluation revealed a clear trade-off between machine learning (ML) and physical high-resolution models in precipitation forecasting. AIFS models achieved the highest skill scores, including SEEPS skill score, Frequency Bias, and Equitable Threat Score, exceptionally when trained with MSWEP and IMERG data. These models also demonstrated the lowest systematic bias across all lead times.
Figure 3: Frequency Bias (top panel) and ETS (bottom panel) from 01 January 2023 to 31 December 2024 for different climate percentiles.
The performance assessment using the Equitable Threat Score (ETS) provided further evidence of the skill of these experimental AIFS models. AIFS (IMERG) and AIFS (MSWEP) achieved the highest ETS values across most percentiles. This indicates their superior ability to forecast precipitation events against a random forecast accurately. In contrast, GraphCast displayed bias patterns like IFS but recorded the lowest ETS skill in the coastal region. IFS itself maintained stable ETS performance across most percentiles. However, like all models, it exhibited a predictable decline in skill for extreme events (i.e., the 95th to 99th percentiles).
Although machine learning models demonstrate statistical superiority, all models, including AIFS and GraphCast, exhibit limited predictability for extreme rainfall events. The physical models, IFS and Destination Earth, more accurately represent the spatial distribution and magnitude of extreme events, likely due to their higher resolution and physical basis. However, these physical models consistently overestimate total rainfall.
The IFS produced a significant excess in accumulated precipitation compared to observed data. Analysis suggests that this overestimation results from large-scale processes rather than convective parameterisation in the mountainous region.
Figure 4: 12 UTC 08 March 2023 Observed precipitation (depicted in circles, rectangles, and triangles), left to right: Total, Convective, Large-Scale Daily Precipitation.
What next for forecasting
In the tropical region of Ecuador, my findings indicate that machine learning models provide superior broad-scale statistical performance, while physical models are better at capturing extreme rainfall events. These evaluation results not only provide insights into the functioning of artificial intelligence and numerical weather prediction (NWP), but also support stakeholders responsible for risk prevention, such as operational meteorologists. By enhancing confidence in model data—including AIFS and IFS—these findings facilitate sustainable development initiatives in Ecuador. Increased trust in these forecasts translates to more effective application to early warning systems, disaster preparedness, and resilience-building at both national and regional levels.
As AI models continue to evolve, they could become powerful tools for early warning systems and disaster preparedness. For Ecuador and many other regions facing climate extremes, that could make all the difference.
Fellowship experience and collaboration
This fellowship between WMO and ECMWF is part of the broader programme which offers specialised opportunities to strengthen capacity in least developed and developing countries. In this role, I collaborated with ECMWF experts in the Evaluation Section, the machine learning team, and the verification team within the Forecasting and Services Department.
I sincerely thank all those who contributed valuable ideas and support throughout this year. I am especially grateful to David Lavers, Gabriel Moldovan, Zied Bouallegue, Becky Hemingway, and Matthew Chantry for their direct support and supervision during my fellowship. Their help was invaluable and without their input, this work would not have been possible. Special thanks to Richard Forbes and the Physical Team of the Research Department. They contributed insightful help in understanding physical processes and expertise in IFS. I also appreciate all ECMWF technical staff for their consistent input throughout the year.