Active: 2012-2014

A Declarative Framework for Data Stream Processing in Pervasive Applications

A stream-processing framework, based upon state-of-the-art stream processing engines, which enables declarative and modular composition of stream processing chains built atop of a set of extensible stream processing blocks. While stream processing blocks are delivered as a standard, yet extensible, library of application-independent processing elements, chains can be defined by the pervasive application engineering team.

The spChains framework supports the elaboration, combination and abstraction of environmental data coming from multiple sources (i.e., sensors) through chains of modular and re-configurable processing elements called stream processing blocks (see Figure 1).

On one hand (Figure 1, right), pervasive applications (listening to events delivered by event drains) are relieved from the heavy burden of data handling, and they only need to define the aggregation and detection patterns (in form of stream processing chains). On the other hand (Figure 1, left), spChains can perform correlation and elaboration of heterogeneous data flows conveyed by the underlying pervasive communication infrastructure (typically wireless sensor networks) and abstracted in form of event sources.

The inner spChains architecture respects the well-known filter and pipe pattern: each component (filter), i.e., each stream processing block, has a set of inputs and outputs. The component reads streams of data on its inputs and provides streams of data on its outputs. A connector (pipe) conveys the stream data from one block output to another block input. The overall data flow starts from a source (event source) and reaches a sink (event drain) through a set of pipes and filters, thus forming an acyclic graph (avoiding convergence issues related to cyclic processing). Event sources and drains abstract sensors and pervasive applications, respectively, defining a standard way (Java interfaces in the provided implementation) of pushing/extracting events in/from the spChains framework, while blocks and chains realize the core data processing.