Computational aspects and performance evaluation of the IFS-XIOS integration

The increment of the spatial resolution for computational Earth system models is nowadays one of the main concerns of the scientific community in order to solve more complex problems, and thus, achieve more accurate solutions to the reality. However, the new complexity requires more computing power that only cutting-edge supercomputers can provide. This requires to use sophisticated HPC techniques to efficiently use the computational resources. In addition, such high resolutions lead to generate an enormous amount of data to meticulously represent accurate solutions.

Current Earth system models usually have inefficient sequential I/O schemes that used to run low grid resolutions, where the generated amount of data for the simulation results was not particularly big. However, sequential I/O schemes do not scale with current models where a lot of parallel resources are used. In order to address this issue, the most adopted approach is to use scalable parallel I/O solutions that offer both computational performance and efficiency.

This document analyzes the I/O process of IFS, one of the most important atmospheric models used in Europe. IFS has two different output schemes, the MF I/O server and a sequential I/O scheme, being the latter the only available in OpenIFS. This sequential scheme is characterized by being inefficient, since all the output is made through the master process. Therefore, here it is presented an easy-to-use development that integrates an asynchronous parallel I/O server called XIOS into IFS. Moreover, different optimization techniques, such as computation and communication overlap, are applied in the integration development to minimize the I/O overhead in the resulting IFS execution.

The results show that the use of XIOS in IFS to output data is certainly good. This new parallel scheme has reduced significantly the execution time of the original sequential scheme. Using the proper configuration, XIOS proves to be a scalable I/O server that keeps a low overhead regardless the amount of IFS processes and the output size. Furthermore, XIOS offers a series of benefits that shorten the critical path of IFS experiments by concurrently running the post-processing task along the IFS execution: data format conversion, online post-processing and CMIP-compliant output. In this scenario, the total execution time is greatly reduced.