This project has ended | -
MISTRAL achieved its key aim of developing an Open Data Portal for multiple types of Italy-oriented meteorological data, including real time forecasts. High-value post-processed forecast products are included, such as those from the "Italy Flash Flood" initiative. As well as being an accessible repository for regional meteorological agencies and other users across Italy, the portal will also assist the new national Italian public Met Service - "Agenzia Italia Meteo" (in its formative stages early in 2023).
ECMWF spearheaded the "Italy Flash Flood" numerical model post-processing and blending initiative. This was a new and innovative approach to predicting, with increased accuracy and (where possible) geographical specificity, local rainfall totals, with a particular focus on extremes that can lead to devastating flash floods. Without post-processing the numerical model forecasts from both high (convection-resolving) and low (convection-parametrising) spatial resolutions tend to fall short of meeting customer needs, for a variety of reasons. The system ECMWF implemented within MISTRAL involved applying the ecPoint post-processing technique to ECMWF ensemble forecasts (18km resolution), and a scale-selective neighbourhood post-processing approach to COSMO ensemble forecasts (2.2km resolution), before blending the two together. Forecasts continue to be generated in real-time to this day (early 2023).
Through the above work ECMWF was able to investigate how to use post-processing and blending to improve forecasts in a critical area, paving the way for potential related future activities as our own model forecasts move to higher and higher resolution (in tandem with initiatives such as Destination Earth = DestinE), and as we adopt a dual resolution approach in operations (in cycle 48r1, due for implementation in 2023, there will be independent daily global ensemble runs at both 9km and 36km resolution). Similarly, ways to improve the quality of ecPoint post-processing, which has been delivering real-time products for several years now at ECMWF, were explored and implemented internally. Verification work showed how the MISTRAL ecPoint-adapted forecasts are extremely competitive, for predicting localised extremes, relative even to a high-resolution ensemble post-processed in a particular way. This is forms part of a gathering body of evidence, from different sources (different high resolution models / data sources / post-processing techniques), of the value and quality of ecPoint post-processing.
Another important benefit of MISTRAL for ECMWF was gaining real-time access to very high density observations across many parts Italy. These are used for both data assimilation purposes, to improve forecasts, and for verification, to measure how good the forecasts are at a local level. Indeed the aforementioned rainfall verification used this observational data.
To find out more see the following:
MISTRAL: Meteo Italian SupercompuTing poRtAL
The project aim is the creation of a National Meteorological Open Data Portal to provide citizens, public administrations and national and international private organizations with meteorological data from observational networks, historical and real-time analyses and forecasts, with high accuracy. The data will be available in terms of gridded fields, probabilistic products (as rainfall forecasts for flash flood prediction) or point time-series coming from the Italian operational forecast modelling chain and post-processing fields (as thunderstorm probability).
The goal of the MISTRAL portal is to facilitate and foster the re-use of the datasets by the weather community, as well as by its cross-area communities, to provide added-value services through the use of HPC resources, turning it into the level of new business opportunities.
Ten specific project’s objectives lead the activity implementation:
- To improve the access to large datasets of the public weather institutions;
- To unleash a large scale weather datasets creative exploitation using supercomputers;
- Big Data Challange: to create a new service at the Italian national level that addresses the scalable exploitation of the massive amount of public open data;
- To exploit supercomputing facilities to analyse complex combinations of large public weather datasets to make forecasts more valuable;
- To improve the interoperability of regional weather data providers based on the adoption of international standards;
- To provide the direct upload of data generated by supercomputers to the Italian National Open Data Portal and to the EU Open Data Portal, assuring trans-European access, interoperability and services linking the weather sector to the core service platform of the EU Open Data Portal;
- To provide official and certified data and services to foster data re-use in accordance with Public Sector Information and open governments initiatives;
- To provide citizens with free access to weather observational and forecast data and visualisations;
- To design and implement new ways of displaying and visualising datasets and metadata, tailored to the user's needs;
- To identify viable business models that can support a sustainable usage of weather datasets and visualisations of Italian public weather institution sources in European repositories, in a trans-national perspective.
Six partners and an inclusive approach to involve users.
The project partners will work together with the national wider meteorological community to promote the “data opening” and to implement tools to collect, harmonise, aggregate, discover, view, download, transform and disseminate scientifically significant weather data and forecast. The consortium of partners will define open data policies for the national weather data in order to define a national regulation and promote the weather data re-use.
In addition to ECMWF, the partners are:
- CINECA, the national facility for supercomputing applications and research, one of the largest infrastructures in Europe, providing HPC cutting edge resources and qualified support to academic and industry organizations (Lead Partner)
- The Italian Civil Protection Department steers the National Service of Civil Protection with a leading role, in cooperation with regional and local governments, for risk prevention, forecasting and monitoring activities as well as for emergency preparedness and intervention procedures in cases of on-going or upcoming crisis events.
- The Regional Agency for the Environmental Protection of Emilia Romagna carries on Numerical Weather Prediction activities at national level and owns a huge monitoring station network and manage a met radar system.
- The Regional Agency for the Environmental Protection of Piedmont owns a huge monitoring station network, remote sensing instruments together with the deep expertise in weather data interpretation, data processing and model forecast verification.
- Dedagroup Public Services, a private company, one of the largest Information Technology group with expertise in systems integration, digital design and digital innovation of companies, public bodies and financial institutions, with deep competencies and experience in geo data, open data, and international standards and directive
Interfaces, services and functionalities
Different interfaces, user experiences, services and functionalities, clear and public data licensing, will be tailored to serve professional users, economic actors and the general public. Both citizens, stakeholders and decision makers will be able to access datasets and metadata, both through the use of open services and also thanks to the availability of visualization tools. Advanced, interactive and innovative visualization solutions will be implemented in order to display the datasets both for scientific and institutional use of data and for non-expert users.
The design of a business model is one of the project outputs: the national portal MISTRAL will be active not only for the duration of the project, but will be a stable instrument for the dissemination of meteorological data and it will be dedicated to different kind of end users, scientific community, researchers, general public, local, national and international institutions. Moreover, this Portal could be integrated into the European Data Portal to achieve a highest level of visibility through a pan-European synergy.
Innovative technology for HPC (High performance Computer) facilities and big data storage will be used to support the MISTRAL portal implementation.
The open portal will be designed for datasets and metadata interoperability, sharing and re-use and will be supported by a large repository for data (gridded data, images, maps, etc.) and metadata. The repository will have in-built functionalities for highly innovative and advanced data and metadata management and will integrate visual analysis components for data visualisation.
Big data storage and HPC facilities will be made available to the project in order to perform post-processing and visualisation of the observed and forecast data, in order to make the forecasts more useful and usable, tailoring the output to the need of the intended audience.
MISTRAL Website
For more information, please visit http://www.mistralportal.eu/.
|
Co-financed by the Connecting Europe Facility of the European Union |