This research line explores the use of advanced stream processing techniques both for the Internet of Things as well as for Data Center Telemetry collection. The explored techniques will be designed to fit Cloud-based approaches as well as Fog Computing solutions.
Summary
Devices and sensors generate streams of data across a diversity of locations and protocols. That data usually reaches a central platform that is used to store and process the streams. Processing can be done in real time, with transformations and enrichment happening on-the-fly, but it can also happen after data is stored and organized in repositories. In the former case, stream processing technologies are required to operate on the data; in the latter batch analytics and queries are of common use. This research line explores runtimes to dynamically construct data stream processing topologies based on user-supplied code. These dynamic topologies are built on-the-fly using a data subscription model defined by the applications that consume data. Each userd efined processing consumes input data streams and may produce output streams that others can consume. A subscription-based programing model enables multiple users to deploy their own dataprocessing services. The runtime does the dynamic forwarding of data and execution of user code on the fly. A public prototype runtime that leverages Apache STORM for parallel data processing has been developed, which combined with dynamic user-code injection provides multi-tenant stream processing topologies.
Objectives
The objectives of this research line include:
- Explore the architecture realtime processing runtimes to allow multi-tenant computation and data-sharing between tenants
- Explore the Lambda/Kappa Architectures for implementing scalable platforms to perform stream-based analytics
- Explore architectures based on Pub/Sub strategies to allow dynamic data-subscription strategies in centralized and distributed physical topologies (Cloud and Fog architectures)
