Partners from the NEXTGenIO project on input/output challenges in supercomputing decided on a hardware architecture when they met at ECMWF on 23 and 24 February 2016.
The project aims to develop innovative solutions to tackle input/output (I/O) bottlenecks in high-performance computing (HPC) as supercomputing moves towards the exascale.
The outcome of the project will be a prototype HPC system designed by Fujitsu with Intel 3D-XPoint Non-Volatile RAM (NVRAM), including newly developed systemware and adapted application stack.
As one of the main application providers, ECMWF is contributing by developing an I/O workload simulator that will be used to evaluate the new HPC system and may be used in future HPC procurements.
ECMWF will also adapt the I/O libraries in its Integrated Forecasting System (IFS) to take advantage of the new NVRAM technologies and minimize I/O impact in daily operations.
“The most urgent item on the agenda was to finalise the hardware architecture to be developed, so that Intel and Fujitsu can perform the detailed design and move on to manufacturing within the timeframe of the project,” said Simon Smart, one of the computer scientists from ECMWF working on the project.
The partners involved in specifying the architecture presented their work to all participants in the project. Intel and Fujitsu then presented details of the proposed architecture and the remaining choices that needed to be made.
The partners unanimously chose one of the proposed architectures and gave Fujitsu the green light to proceed with their design work.
Participants came from the eight NEXTGenIO partner organisations.
Representatives from the Barcelona Supercomputing Centre, the Technische Universität (TU) Dresden and Allinea presented their plans for novel scheduling technologies and new mechanisms to distribute data and computation around HPC systems.
ECMWF presented a range of possible architectures to adapt its operational forecast process to make use of NVRAM technology. This would feature a new I/O layer that minimises file system access within the time-critical window, whilst still providing a resilient infrastructure.
ECMWF also presented the design of the I/O workload simulator. In particular, the metrics to be used to measure a real workload on an HPC system were discussed with representatives from Allinea and TU-Dresden. A work plan was agreed for how to consider modelling the measured workload.
The project partners will now focus on their respective areas and will meet again in May or June to discuss progress, before the first technical review by the European Commission in July.
NEXTGenIO started on 1 October 2015 and is set to run for three years. It is a Horizon 2020 EU-funded project with an 8.1m-euro budget and is co-ordinated by the Edinburgh Supercomputing Centre (EPCC).