Members of the Storage System group at Barcelona Supercomputing Center (BSC) are actively participating in the IOStack, a European project which began in January 2015. The aim of the project is to create a software-defined storage (SDS) toolkit for Big Data on top of the OpenStack platform. BSC’s main tasks in IOStack are in the SDS Toolkit and Data Services work packages. The group will work on areas relating to the performance (bandwidth, latency) control of the SDS toolkit control plane. In addition, the group plans to introduce several data services with features like deduplication, compression, or caches going from hot data cache to intelligent prefetchers and filters. Finally, the group will contribute to the Architecture, Prototyping and Validation work package, as the group has knowledge of lower layers (input/output (I/O) path) and will support the consortium on these topics.
“Software-defined storage is a management and virtualisation technique with great potential, although it is still in its infancy. The IOStack project will develop this concept and adapt it to Big Data, one of the areas where this kind of approach will be most needed, given the I/O requirements of Big Data applications. The overall aim of the project is to offer SDS for Big Data and thereby enable data analytics as a service, making it affordable to businesses of all sizes,” says Toni Cortés, Storage Systems Group Manager at the Barcelona Supercomputing Center.
The BSC Storage System research group has extensive experience in data management at all levels, from the node level to the wide-area distributed level. At the node level, the group has contributed to new file systems and optimisations of the I/O stack. At the cluster level, it has made several contributions to cooperative caches, as well as to heterogeneous and scalable storage systems. Finally, at the wide-area level, the group has participated in the definition and implementation of a world-area file system.
IOStack is designed for deployments of data analytics in virtual environments. Virtual environments allow very flexible deployment of analytics frameworks but deliver lower performance than bare-metal deployments. IOStack will focus on how Software Defined Storage can use its knowledge of Cloud topology and the real-time dynamic characteristics of the Cloud to deploy analytics jobs that will run and complete within guaranteed service-level agreements (SLAs) and timescales.
IOStack will enable efficient execution of virtualised analytics applications over virtualised storage resources thanks to flexible, automated, and low-cost data-management models based on software-defined storage (SDS). In order to achieve this, IOStack will also work on the following areas: storage and compute disaggregation and virtualisation, SDS Services for Analytics and orchestration and deployment of big data analytics services. The IOStack project is formed by a consortium of several European industrial and research partners including University of Rovira i Virgili, IBM, MPSTOR, Eurecomand the Barcelona Supercomputing Center.
More information: http://www.iostack.eu/