StreamCruncher 2.1 RC

StreamCruncher is an Event Processor. The project supports a language based on SQL which allows you to define Event Processing constructs like Sliding Windows, Time Based Windows, Partitions and Aggregates.
Queries can be written using this language, which are used to monitor streams of incoming Events. StreamCruncher is a multi-threaded Kernel that runs on Java.
Ever since RFID became commercially viable and companies started packaging their entire product line as Business Activity Monitoring Stacks, an old idea that has been around for almost a decade is now seeing the light of day. A handful of Startups have begun turning this Idea of Complex Event Processing (CEP) and Event Stream Processing (ESP) into Commercial grade software. Several people have contributed ideas to this field. The most frequently mentioned name would be Prof. Luckhams. Prof. Widoms contribution to the field is also noteworthy.
Those in a hurry can skip to this section.
Now, back to CEP and ESP. An over-simplified definition would probably look like this - Assume that a System produces "Events" that describe the state of that System at those instants. Slicing and Dicing the "live" Stream of Events (in near real-time) can reveal vital information. Since this information is available on the live data and not stale data, it can be used to look for Warning signs, implement JIT materials sourcing, Automated Trading and other highly nimble systems.
Events from multiple sources can be co-related to reveal information about their interactions and dependencies. CEP and ESP are not exactly the same, but well leave that to the Scholarly Papers to disambiguate. All this does not seem new at all to people whove been working on Control Systems. Data Warehouse specialists and Rule Engine experts might start mumbling something about "Old Wine..".
However, only lately are we seeing generic, off-the-shelf Software focusing on CEP and ESP. Each one of those Products or Projects distinguish themselves in Performance (Real-time, Soft Real-time), Ease of Integration (Features, Mild to Steep Learning Curves, Manageability), Speciality (Automated Trading Platforms) or General purpose stacks etc. Or, perhaps all of them, to varying degrees.
For a Program to continually Slice and Dice an Event Stream and thereby, discern/infer something useful about the state of the System before it goes stale, requires a variety of Technologies to converge. Weve been using SQL to handle Transactional Data stored in Databases, Data Warehouses to analyse Terrabytes of Offline data, Rule Engines to infer from a large collection of constantly changing Facts.
ESP and CEP Systems provide building blocks using which Users can build an intelligent System, with active feedback. All CEP and ESP Systems provide a mechanism by which Queries/Conditions/Criteria can be provided that are constantly evaluated against the Event Stream. Some systems provide drag-n-drop GUIs to specify such "Queries", while some have their own proprietary Query syntax. In essence, they provide a Domain Specific Language to encode such Queries/Rules, a facility to pump Events into the System and a way to channel the Output Stream to do something useful.
Enhancements:
- The pre-filter for input event streams now supports <, >, !=, =, *, /, +, -, "in (..)", "not in (..)", "and", and "or".
- The "in" clause can refer to an SQL sub-query.
- Such sub-queries are cached by the kernel to improve performance.
- An additional property, cacherefresh.threads.num, can be configured to specify the number of sub-query cache processing threads to use.
- Two new test cases have been added to test the new features: H2StartupShutdown3Test and ThreeEventOrderTest.