One of the critical steps of any HPC program is reading data and storing temporal or final results to or from the file system. Knowing how and when to read or write into a file is essential because it is one of the most time-consuming operations. Not only is it vital to be efficient when doing IO operations (input-output), but it is also crucial to how the data is accessed and how to keep it. Another critical decision is how many processors will execute these IO operations and how they will communicate. In this case, these tests have been executed on gpfs/projects on Marenostrum 4. We will use a pure MPI communication scheme and HDF5 1.10.1.
Considering this, we have performed several tests using the h5bench benchmark to see the performance and impact of different memory access patterns and temporary or end-file storage patterns using a set of data structures.