spChains

Active: 2012-2014

A Declarative Framework for Data Stream Processing in Pervasive Applications

A stream-processing framework, based upon state-of-the-art stream processing engines, which enables declarative and modular composition of stream processing chains built atop of a set of extensible stream processing blocks. While stream processing blocks are delivered as a standard, yet extensible, library of application-independent processing elements, chains can be defined by the pervasive application engineering team.

The spChains framework supports the elaboration, combination and abstraction of environmental data coming from multiple sources (i.e., sensors) through chains of modular and re-configurable processing elements called stream processing blocks (see Figure 1).


 

On one hand (Figure 1, right), pervasive applications (listening to events delivered by event drains) are relieved from the heavy burden of data handling, and they only need to define the aggregation and detection patterns (in form of stream processing chains). On the other hand (Figure 1, left), spChains can perform correlation and elaboration of heterogeneous data flows conveyed by the underlying pervasive communication infrastructure (typically wireless sensor networks) and abstracted in form of event sources.

 

The inner spChains architecture respects the well-known filter and pipe pattern: each component (filter), i.e., each stream processing block, has a set of inputs and outputs. The component reads streams of data on its inputs and provides streams of data on its outputs. A connector (pipe) conveys the stream data from one block output to another block input. The overall data flow starts from a source (event source) and reaches a sink (event drain) through a set of pipes and filters, thus forming an acyclic graph (avoiding convergence issues related to cyclic processing). Event sources and drains abstract sensors and pervasive applications, respectively, defining a standard way (Java interfaces in the provided implementation) of pushing/extracting events in/from the spChains framework, while blocks and chains realize the core data processing.


 Technical reports

Configuration samples

  • Sample configuration including all blocks [.xml]
  • Sample configuration including the and block [.xml]
  • Sample configuration including the range block [.xml]
  • Sample configuration including the join block [.xml]

 


 

Downloads

v2.0 - (RUNTIME CRUD)

Older releases

v1.5 - (runtime chain update)

  • spChains library (core + block implemementation based on Esper)
  • Required libraries
  • source code (Licenced under the Apache v2.0 Licence, except for libraries released under other Open Source licensing schemes)
  • spChains test program (sources)

v1.1

  • spChains library (core + block implemementation based on Esper)
  • Required libraries
  • source code (Licenced under the Apache v2.0 Licence, except for libraries released under other Open Source licensing schemes).

v1.0

  • spChains library (core + block implemementation based on Esper)
  • Required libraries
  • source code (Licenced under the Apache v2.0 Licence, except for libraries released under other Open Source licensing schemes).