Putting users first: the common language behind ECMWF’s data

20 July 2026

Stylised digital globe overlaid with computer code and data streams, representing artificial intelligence, data processing, and global computing technologies.

Image: © monsitj / Adobe Stock

Authors: Simon Smart, Umberto Modigliani

ECMWF takes an unusual approach to data management. At first glance, it may seem driven by a substantial technical legacy – after all, ECMWF has been producing operational forecasts since August 1979, when the technical landscape looked very different.

In reality, however, ECMWF’s approach has always been driven by people's needs.

For more than forty years, this philosophy has been expressed through a shared way of describing data, known as the MARS language. Rather than asking users to understand where data is stored or how systems are organised, the MARS language enables them to describe the data they need using scientifically meaningful terms. The underlying systems handle everything else.

As forecasting has evolved – from multi-node supercomputers to cloud computing and artificial intelligence (AI)-based systems – this language has provided a stable foundation for an increasingly complex data ecosystem.

Designing systems around people

ECMWF works with a diverse community of users. Meteorologists, hydrologists, climatologists, and other Earth-systems scientists produce, consume, and share data, and write software to interact with it. Most are scientists rather than software engineers and wish to describe the data they produce and need using scientific terms.

At the same time, downstream developers in our Member and Co-operating States, as well as in commercial organisations, often lack specific knowledge of ECMWF’s internal systems – but they do want long-term, stable, easy-to-understand interfaces to our operational systems.

For the general public, the technical details are even less relevant. They simply want to find out what the weather will be like, or what it was like, in clear, human terms.

These users all have very different needs and access patterns. Needing to know when the forecast is available is very different to wanting to know tomorrow’s forecast for a specific country, or comparing the weather to ten years ago.

But all of these uses can be described in human-understandable, scientifically meaningful terms. The MARS language was developed to do exactly that.

It provides a common, semantically and scientifically based vocabulary for describing data, creating a shared language that connects people, software and infrastructure across ECMWF's entire data ecosystem.

"Technology changes quickly, but the scientific questions our users ask are often the same. The MARS language allows us to preserve that scientific way of thinking while continuously modernising everything underneath," said Simon Smart, Team Leader in the Forecasts and Services Department at ECMWF.

A different data philosophy

Most software developed in academic research environments is designed to solve a specific and pressing scientific problem as quickly as possible. Over time, this leads to systems with strategically named files, complex directories, and file and object names, as well as incrementally developed conventions.

As these systems become more complex, these choices tend to restrict changes to workflow structure and data layouts and require developers and users to have technical knowledge of how and where data is stored.

Unlike most organisations, ECMWF has taken a different approach in which users do not need to know where or how their data is stored. Instead, users simply describe the data they need. The underlying systems then make the relevant decisions, determining where the data resides and how it should be processed and delivered.

This approach provides significant benefits, both technical and human. Technically, it decouples the software and the structure of ECMWF workflows from the way data is stored and manipulated. For humans, it provides a scientifically intuitive way for users to engage with data. It also clearly separates the roles of scientific data users, workflow integrators, compute resource administrators and data store administrators, allowing specialists to excel in their own domains whilst providing a common language for them to communicate with each other.

“By separating scientific intent from technical implementation, we've been able to evolve our infrastructure continuously while keeping the user experience consistent," said Tiago Quintino, Head of the Development Section in the Forecasts and Services Department at ECMWF.

The MARS language

The MARS language is the vocabulary that makes this possible. Each piece of data at ECMWF has a unique identifier, which takes the form of a set of key-value pairs, where each keyword is assigned a value that defines an aspect of that data. These identifiers can be constructed to describe both the data that will be provided and the data that is desired.

With some domain-specific knowledge, these key-value pairs are largely self-explanatory. They describe the dataset, the type of data (e.g. atmospheric forecast), the parameter (e.g. temperature), where it exists (e.g. level in the atmosphere), and when it is valid (forecast time and step).

When accessing or processing data, a request is constructed by combining a verb with a selection. Each keyword can take one or more values – for instance, requesting both geopotential and pressure using param=Z/T.

Where multiple values are provided for multiple keywords, the system describes all possible combinations. This can be used to describe any arbitrary action and a set of data to operate on.

Example of a MARS language request showing data description fields, processing arguments (grid and area), and target output location for retrieving meteorological data.

A sample dissemination request, showing the verb ('disseminate), data description, verb-specific processing arguments and I/O descriptor.

ECMWF has been using this object-based approach for many years – long before it became popular for facilitating massive scaling of web infrastructure at a global scale. This is because an object-based approach permits consideration of vast numbers of units of data described by identifiers which do not map directly to underlying (technical) storage locations.

The cost of consistency

Using the MARS language across our systems to describe data is very powerful. It provides a consistent and stable way to describe data and an interface for developers to work with. But it comes at a cost – all data and all systems must be engaged with this language.

This means we require a governance process to determine the vocabulary used to describe data and to decide how the language should handle new developments and new science. Scientists cannot release new data into our systems until it has passed through this process. Similarly, when integrating third-party models, adapters are needed to translate their data. Without careful stewardship, inconsistencies would gradually accumulate, making systems harder to maintain and reducing interoperability between services.

Maintaining the language also requires organisational buy-in and sustained resource commitment over decades. Decisions made today need to support technologies and developments that may not emerge for many years.

“Good governance isn't about slowing innovation – it's about making sure today's innovations remain usable ten or twenty years from now. Every addition to the language needs to work not just for one project, but for the whole ecosystem,” said Simon.

Timeline titled “The Origin of an Ecosystem” showing the evolution of ECMWF’s MARS language from the late 1970s to the 2020s, highlighting key milestones including Genesis, the MARS archive, parallel computing, system-wide expansion, scaling of data services, and support for AI and cloud technologies.

Evolution of the MARS language and data ecosystem, from ECMWF’s first operational forecast in 1979 to support for AI and cloud-based services in the 2020s.

Through the looking glass – a glimpse into the future

The value of a common, semantic approach to accessing data becomes clearer as ECMWF moves into an era of AI-driven forecasting, with rapid iteration of the models, workflows and data in use.

Machine learning (ML) training datasets can now be described and assembled using configurable recipes, and the newly trained models can directly express the data they make available to downstream systems.

For downstream data users, everything is handled using the common MARS language, so they do not need to change anything other than specifying that they wish to use the AI datasets.

“One of the biggest advantages of a common language is that new technologies don't have to reinvent how they interact with the rest of the ecosystem,” said Simon.

The MARS language gives us tools to build other infrastructure for the future. We operate in an increasingly connected, multi-data centre world, and the MARS language provides the expressiveness of ECMWF data availability notifications and serves as the basis for our data transfer and synchronisation systems.

This capability is essential for generating new data and serving new users. Upcoming products such as ERA6, alongside increasingly high-resolution forecast data, will be game-changers for societal understanding of weather and climate impacts. However, supporting these data requires tooling to handle the increased load and workflows.

To meet this challenge, we are building advanced systems, including Polytope, GribJump and a Zarr adapter layer, to make regional and cloud-based access to data easy and efficient. Although these systems address different challenges, they are all built using the MARS language.

For users, consistency is key. Whether they are conducting scientific research, running analytics, training models, or telling stories with our data, they interact with it in the same way. They describe the data they want, and the underlying systems take care of the rest.

“As ECMWF’s data volumes, technologies and user communities continue to grow, the MARS language gives us something essential: continuity for users. It allows us to modernise our infrastructure without asking people to relearn how to access the information they rely on,” said Umberto Modigliani, Acting Director of Forecasts at ECMWF.

Making these datasets as accessible to the public as possible will maximise ECMWF's positive social impact. By providing a consistent way to discover and access information, the MARS language allows scientists, developers and decision-makers to focus on extracting value from the data rather than navigating the underlying infrastructure.

Final outlook

ECMWF has, for many years, built its data systems around a semantic, metadata-driven object storage approach. We have found this approach adaptable, able to accommodate new requirements as they emerge. Over time, as systems have evolved, the MARS language has remained a common thread, linking our technical systems in a way that is accessible to those who engage with them.

Maintaining this approach has required sustained investment and organisational commitment, but over the long term it has paid off enormously. ECMWF is proud of the MARS language, despite its quirks, awkwardness and legacy, for the long-term practical and scientific pay-off it has delivered.

“The MARS language isn't something most users ever think about, but you could say that it’s been part of the secret sauce of ECMWF's success,” concluded Umberto.