When a tropical cyclone is developing or a slow-moving rain system is building over a river catchment, the value of a forecast depends critically on how quickly it arrives.
Emergency managers and national meteorological and hydrological services are relying on our forecasts to help make decisions about warnings, evacuations and resource deployment, often under significant time pressure. For these situations, minutes really do matter.
This is the operational reality that shapes everything about how ECMWF disseminates data. The Centre runs one of the largest operational meteorological data dissemination services in the world, delivering over a hundred terabytes of forecast data to over a thousand users, four times a day, every day. Much of it is consumed within minutes of generation.
With so much depending on it, the infrastructure that moves data is not a background concern; it is an essential part of the forecast service itself.
ECMWF – one of the world’s largest handlers of weather and environmental data
ECMWF’s forecasts rely on huge quantities of observations from across the globe, which feed advanced numerical and AI weather forecasting models.
Our world-leading forecasts feed into downstream products consumed by national meteorological and hydrological services (NMHSs), commercial partners, researchers, and public-facing services across every continent.
And ECMWF’s remit extends beyond weather forecasts. The Centre is a key producer and distributor of climate and environmental data through its role in the EU’s Copernicus Earth observation and Destination Earth programmes.
While ECMWF’s Member and Co-operating States remain the Centre’s core users, the number and diversity of users have grown rapidly. When this is coupled with the move to ever higher resolutions and a growing portfolio of products, ECMWF’s forecast dissemination task represents a huge and growing data challenge.
Three hundred terabytes, three delivery channels
It is worth spelling out the breadth of ECMWF’s data dissemination challenge:
Real-time forecasts via ECMWF’s Production Data Store (ECPDS) (~100 TB/day): The core operational output. High-resolution deterministic and ensemble forecasts, produced on a fixed schedule, delivered under tight timeliness constraints to NMHSs and commercial partners.
Open data (~100 TB/day): Following ECMWF's transition to open data (under CC BY 4.0) in October 2025, a wide range of forecast products is now freely available. This stream serves the growing community of researchers, startups, and downstream service providers who depend on open access to global forecast data. ECPDS serves approximately one third of the full 100 TB/day with third-party clouds providing the remainder.
Climate Data Store (~100 TB/day): Products like ERA5 and ERA6, through our role in the Copernicus Climate Change Service (C3S), provide consistent, gridded records of past weather and climate conditions spanning decades. Through our role in the Copernicus Atmosphere Monitoring Service (CAMS), we deliver near-real-time air quality forecasts, greenhouse gas monitoring, and aerosol products.
Each stream has different latency requirements, different access patterns, and different user communities – but all depend on the same underlying commitment to reliable, timely delivery.
The ECPDS – ECMWF’s data workhorse
ECMWF’s Production Data Store (ECPDS) sits at the heart of ECMWF’s dissemination infrastructure – delivering critical forecast products with the exceptional reliability and speed demanded of an operational 24/7 weather forecasting centre like ECMWF.
The ECPDS is not just a single system with a single role, for example it handles both the acquisition of observations flowing into ECMWF and the dissemination of forecast products to over a thousand users across multiple networks.
ECMWF’s Production Data Store (ECPDS) sits at the heart of our real-time forecast dissemination system.
The software is also open source and, for a long time, has been available to Member and Co-operating States to use for their own data flows.
ECPDS is engineered for low-latency delivery. For time-critical products – the fields that national weather services need to initialise their own models or trigger warning systems – dissemination targets are measured in minutes.
For Member and Co-operating States, and some commercial users, products are released as soon as they are generated. For other users, release follows the standard dissemination schedule – which still represents a rapid and reliable service for the majority of use cases.
There are situations where ECMWF has made a deliberate policy choice to prioritise immediate release for everyone. Key tropical cyclone products, for example, are made available as soon as they are produced – a recognition that in rapidly evolving high-impact situations, the sooner a forecast is received the better.
Efficiency without hyperscaling
ECMWF’s core dissemination infrastructure is purpose built and operated in-house – a deliberate decision for workloads that are sustained, high-volume, and latency-sensitive around the clock. For that kind of constant operational demand, bespoke infrastructure tuned to scientific workloads offers reliability and performance characteristics that are difficult to replicate on general-purpose platforms.
Cloud platforms also play a role in how ECMWF data reaches its users. AWS and Azure both host mirrors of ECMWF's open data, dramatically expanding the reach and accessibility of forecast products for the research and commercial communities that depend on them.
ECPDS itself is already extending into cloud environments by deploying data movers (referred to as continental data movers) on cloud platforms, strategically located closer to users to reduce latency, improve transfer performance, and better handle growing data traffic.
As data volumes continue to grow toward the petabyte scale, the question of how cloud infrastructure fits into ECMWF's broader dissemination architecture is an open and active one. The goal, as it has always been, is to get the right data to the right users as quickly and reliably as possible – and the infrastructure choices that best serve that goal will continue to evolve.
Ensuring reliability and fair access
Delivering such vast and varied data streams requires robust mechanisms to maintain performance and fairness.
Fair access, robustness and performance under constant high load across both operational and climate-scale production are essential:
Queuing and throttling: ECMWF employs intelligent queuing systems and throttling controls to manage demand, ensuring that time-critical products are prioritised and delivered promptly.
Prioritisation and quality-of-service: data flows are categorised by urgency and importance, with operational forecasts receiving top priority. Quality-of-service mechanisms guarantee that essential users, such as national meteorological services, have reliable access even during peak demand.
Fair-use policies: to prevent congestion and ensure equitable access, ECMWF enforces fair-use limits, balancing the needs of diverse users without resorting to hyperscale infrastructure.
Reducing friction, not just adding bandwidth
Delivering data fast is necessary but not sufficient. If users struggle to find, access, or interpret the data once it arrives, speed alone does not solve the problem.
Points of ‘data friction’ can compound. A dataset can be openly licensed and delivered with low latency, but still present significant barriers if documentation is poor or tooling is limited.
One aspect of ECPDS that aims to reduce friction is enabling users to see their own data delivery information, such as network speed and file availability.
The technical details
The key components that support the ECPDS are housed in ECMWF’s data centre in Bologna with duplicate systems in each data hall to prevent downtime and data loss.
An overview of the physical infrastructure supporting ECMWF’s Production Data Store (ECPDS).
The system stores data as self-contained objects with a unique ID and metadata, which provide a detailed description of the data. This style of data system offers unlimited scalability (so allows for growing data volumes), suits the diversity of data that ECMWF deals with, provides faster access and allows easier replication across multiple locations (giving greater security and robustness) – all essential to support ECMWF’s critical operational forecasting role.
The Climate Data Stores service, developed and operated by ECMWF under the EU Copernicus Earth observation programme, is underpinned by ECMWF’s Common Cloud Infrastructure. Data are open and free. Users can access data through the Copernicus Climate Change Service (C3S) Climate Data Store and the CAMS Atmosphere Data Store interfaces.
Why this matters for data users
If you are consuming ECMWF data operationally, the dissemination architecture is not an abstract concern. It determines when your data arrives, how reliably it arrives, and what happens to your access during high-demand periods.
Understanding the prioritisation model helps explain why real-time products arrive with consistent timeliness while bulk archive requests may queue during peak windows. Understanding the fair-use framework helps explain the rate limits and access tiers that shape how you interact with ECMWF's data services. And understanding friction factors helps identify where the real bottlenecks are — which may not always be where you expect.
The value of ECMWF’s data dissemination is evident in real-world scenarios.
The risks from extreme weather events are growing as the climate changes. Rapid dissemination of forecasts enables timely warnings and response, saving lives and property.
For example, Europe was hit by storm Boris in September 2024. ECMWF forecasts delivered via ECPDS coupled with flood forecasts from the Copernicus Emergency Management Service provided effective warnings – with indications of high flood risk almost a week in advance. Coupled with other sources of information, this allowed national meteorological and hydrological centres and other civil authorities within Member and Co-operating States to issue weather, flood and other emergency warnings at the national and regional level.
As another example, CAMS provides daily global analyses and forecasts of atmospheric pollutants and background air quality for the European domain. These forecasts can be used as they are, but they also serve as input to a wide range of downstream services, such as national air quality forecasts, smartphone applications and policy tools reaching millions of users.
What does the future hold?
Our systems will continue to evolve, as they have since ECMWF was established over 50 years ago, growing continuously while remaining focused on pushing data to meet the needs of our Member and Co-operating States and other users.
ECPDS is a long-established application. It continuously adopts modern technologies and follows a perpetual cycle of improvements to benefit from performance gains, scalability, and evolving user requirements.
We will see the introduction of new services, based on our polytope service, which will allow more flexible, on-demand access to products based on a "pull" paradigm. Polytope provides novel ways to extract and manipulate data, allowing users to read and retrieve only the minimum amount of data they need.
ECMWF's approach to dissemination will continue to reflect a broader principle: that operational weather data is a time-critical infrastructure, and the systems that deliver it should be engineered with the same rigour applied to the forecasts themselves.
Further reading
This article is part of ECMWF’s In Focus series on data, exploring how evolving infrastructure, open data, and AI-ready systems are reshaping access to weather and climate information:
- Data friction and the user experience: a framework for improvement
- The living archive: inside ECMWF's exabyte-scale meteorological data repository
- Powering the AI weather revolution: from ERA5 to AI-ready pipelines
- ECMWF's data infrastructure and services: ready for the era of Common European Data Spaces