February 12, 2014

Operational Business Intelligence for the Reactive Enterprise

We are in a day and age where infrastructure, and to some extent, businesses, are moving towards reactive IT instead of traditional proactive IT. Why reactive IT? Think auto-scaling vs traditional capacity planning for instance. Or the move towards schema-less NoSQL instead of traditional RDBMS. All in all, reactive IT enables an enterprise and its infrastructure to react to its internal and external environment. In order to achieve this, a real-time, or near real time, event driven model is a must –  agents to pull/push streams of events, systems to process these events quickly and efficiently and the ability to use these processed events to react.

Operational Business Intelligence (knowns as OBI or operational BI) or Operational Intelligence is defined as the analysis of operational data and information in an enterprise – this deals with the real time or low latency analysis of streaming events or batched enterprise data providing feedback in terms of input to the enterprise. OI provides organisations with real time insights into enterprise operations, providing organisations the ability to act upon, or in other words react, to events – enabling organisations to ‘listen’ to and process events as they come in, detect anomalies and patterns, and take reactive measures.

Of course, with large amounts of data come big challenges, and with big challenges come big data. With the possibility of dealing with massive data sets within OI, big data concepts have started becoming a mainstay in organisational OI. Modern OI solutions have started focusing on large NoSQL data stores possibly processed in batch mode and streaming events.

OI would provide a unified, correlated view of streaming big data, processed big data, complex events and processes, with the ability to analyse, mine and process data and information – a prerequisite to building a reactive enterprise. Enterprise users and devops have come to expect the following kind of information from a unified OI view

  • Analysis of information, and mining for patterns
  • Adhoc, real time dashboards
  • Adhoc search of patterns across the enterprise
  • Alerting based on event occurences
  • Monitoring of system health, load, KPIs


OI has many technology components, often with shared feature sets. Some of the notable solutions are

  • Business Activity Monitoring (BAM) – Monitoring of activities and events, usually batch processed and provided via dashboards. Used to track  KPIs related to activities and performance.
  • Complex Event Processing (CEP) – Processing of a continuous stream of events, usually in-memory. High performance with the ability to detect certain patterns and anomalies in the incoming streams.
  • Business Process Management (BPM) – Model driven execution of processes and policies, including business workflows with human intervention.

Independently the above components handle a very specific area of OI, and together they provide the building blocks for a comprehensive and unified OI.

In a later post, we will explore how the WSO2 stack, a comprehensive Open Source middleware stack, can be utilised for Operational Intelligence in an enterprise.


May 2, 2013

Real time Complex Analytics for Football

Filed under: Development,WSO2 — mifan @ 6:54 pm
Tags: , ,

Srinath recently wrote about the use of WSO2 CEP for the ACM DEBS 2013 challenge on Real time Complex Analytics which was a real eye opener. The challenge, enabled by sensor networks on player’s shoes, the goalkeeper and the football, was to conduct real time analytics on a football game, and provide useful analytics and real-time reports to the managers.

The post describes the usage of WSO2’s Complex Event Processor (CEP) which was used to implement the use cases of the challenge – namely Running analysis of the players, Ball possession analysis, a heatmap of player locations at various times and shots on goal analysis. CEP is a high performance and scalable event processor that can read streams of ‘data’, extract meaningful events and process them real time in memory – this means the ability to process large amounts of events, fast! In this case, continuous steams of data and events would be fed into the system via the various sensors, at a rate of 15,000 position events per second, whilst the player sensors and the ball sensors output events at a rate of 200MHz and 2000MHz respectively. According the blog, WSO2 CEP processed 50,000 events per second, which is quite impressive.

Just imagine the possibilities such an implementation can provide to the game – I’m awaiting the day when the TV alerts me, possibly 2 seconds before the event, that Robin Van Persie’s shot would have a 99% chance of finding the back of the net beating Petr Cech – based on analytics of the kick (the curvature , wind speed, rotation) and analytics of the defense (the distance of the goalkeeper and probability of him reaching the ball based on historical data, the distance between defenders). Or based on Theo Walcott’s speedy run and Carzola’s immaculate pass, as well as the position of the defense and the keeper, Walcott would end up with the ball beyond the defense in an on-side goal scoring position in the next 3 seconds – the mini siren on the TV goes off, telling me to watch the screen for the next 5 seconds (of course, that is assuming my reaction times are good, let alone the goal keeper’s). And I can hear us hard-core fans of football saying – “who would take their eyes off a football game anyways?”, but what if!

Minority Report for football, anyone?

XKCD Future Comic

Future (Source: XKCD)

Create a free website or blog at