The Workflows and Distributed Computing team at the Barcelona Supercomputing Center is proud to announce a new release, version 2.1 (codename Bougainvillea), of the programming environment COMPSs.

This version of COMPSs, available from today, updates the result of the team’s work in the last years on the provision of a set of tools that helps developers to program and execute their applications efficiently on distributed computational infrastructures such as clusters, grids and clouds. COMPSs is a task based programming model known for notably improving the performance of large scale applications by automatically parallelizing their execution.

COMPSs has been available for the last years to the MareNostrum supercomputer users and to the Spanish Supercomputing Network and has been adopted in several research projects such as OPTIMIS, VENUS-C, EUBrazil OpenBio, EUBrazil CloudConnect, transplant, EGI and ASCETIC. In these projects COMPSs has been applied to implement use cases provided by different communities across diverse disciplines as biomedicine, engineering, biodiversity, chemistry, astrophysics and earth sciences. Currently it is also under extension and adoption in applications in the projects EUBRA BIGSEA, NEXTGenIO, MUG, mf2C, TANGO, the CoE BioExcel, the EXPERTISE ETN, and the Human Brain Project flagship.

The new release comes with new features such as a more integrated mechanism for those tasks that invoke binaries. This extension enables a better integration with programming models such as MPI or OmpSs. In this sense, a COMPSs workflow can be now composed of sequential tasks, multi-threaded tasks (for example, a task that internally is parallelized with OmpSs) and multi-node tasks (a task that internally is parallelized with MPI). The scheduling has been extended accordingly to support this variety of tasks.

COMPSs has been extended with a better support for linear algebra kernels (especially for those based on calls to the Intel MKL library) and two-levels of parallelism can be combined: task-level and thread-level; obtaining very competitive performance results in large number of nodes of MareNostrum.

The Python binding has been improved with a persistent worker that reduces the overhead of starting a new Python interpreter for each task, which benefits applications with fine grain tasks. Also, the integration with the Jupyter notebook benefits a large number of users from communities that use this environment for their Python codes which will be able to run them in parallel with PyCOMPSs.

The interoperability with containers platforms, that already includes Docker and Mesos, has been extended in this new release with the inclusion of Singularity, in order to support HPC clusters. COMPSs 2.1 comes with other minor new features, extensions and bug fixes.

COMPSs has had around 1000 downloads last year and is used by around 20 groups in real applications. COMPSs has recently attracted interest from areas such as image recognition, genomics and biodiversity, where specific courses and dissemination actions have been performed.

During last years, the team efforts have been focusing on the nowadays-emerging virtualization technologies, adopted by cloud environments. In such systems, COMPSs provides scalability and elasticity features by dynamically adapting the number of resources to the actual workload.

COMPSs is interoperable with both public and private cloud providers like Amazon EC2, Google Cloud, OpenNebula and with OCCI compliant offerings.

The packages and the complete list of features are available in the Downloads page. A virtual appliance is also available to test the functionalities of COMPSs through a step-by-step tutorial that guides the user to develop and execute a set of example applications.

Additionally, a user guide and papers published in relevant conferences and journals are available.

For more information on COMPSs please visit our webpage: http://www.bsc.es/compss

More Info about Barcelona Supercomputing Center

Barcelona Supercomputing Center (BSC) is the national supercomputing centre in Spain. BSC specialises in High Performance Computing (HPC) and its mission is two-fold: to provide infrastructure and supercomputing services to European scientists, and to generate knowledge and technology to transfer to business and society.

BSC is a Severo Ochoa Center of Excellence and a first level hosting member of the European research infrastructure PRACE (Partnership for Advanced Computing in Europe). BSC also manages the Spanish Supercomputing Network (RES).

The Workflow and Distributed Computing team at the Barcelona Supercomputing Center aims to offer tools and mechanisms that enable the sharing, selection, and aggregation of a wide variety of geographically distributed computational resources in a transparent way. The research done in this team is based in the former expertise of the group, and extending it towards the aspects of distributed computing that can benefit from this expertise. The team at BSC has a strong focus on programming models and resource management and scheduling in distributed computing environments.