LOCA Series: "Enabling high-level parallel programming on multi-FPGA clusters"

Field Programmable Gate Arrays (FPGA) are still relatively new in the High Performance Computing (HPC) field. Hence, they still lack a mature ecosystem that allows non-FPGA experts to scale an application with many devices operating in parallel. In this presentation, I will talk about how we added support for message passing inspired by the Message Passing Interface (MPI) to the Marenostrum Exascale Emulation Platform (MEEP) cluster.

I will introduce the OmpSs@FPGA programming model, which allows C/C++ code to run on the FPGA and call functions that behave like the well-known MPI\_Send/Recv. I will also give details about how to implement the message passing runtime over the MEEP 100Gb Ethernet network, as well as the challenges I found during development. On the software side, I will show how to manage FPGAs that are PCIe-hosted by remote CPU nodes. I.e. from any CPU node we can load the bitstream, configure it, and transfer data to FPGAs that are attached to a different node. Finally, I will present the evaluation results in the MEEP cluster: Bandwidth of FPGA-FPGA, and CPU-FPGA local and remote communication, as well as the performance of benchmarks scaling from 1 to 64 FPGAs. The benchmarks are N-body, Heat with Gauss-Seidel solver and Cholesky decomposition. Comparing the results with the MareNostrum 4 supercomputer, we get 2.3x and 3.5x better performance per power for N-body and Heat.