Data arrives at very high velocity


High Velocity Big Data / Application Scenarios

21 May , 2015  

Today I gave a talk at the annual IDC DataHub event in Vienna at the Austria Trend Hotel Park Royal Palace. The talk covered basic technology available for high-velocity, streaming Big Data. We discussed streaming systems in general and some implementation details of Apache Storm in detail.

As the Big Data crowd already knows in Big Data mainly 3V’s are addressed. The Volume, Variety, and Velocity of data. (OK IBM believes that we also need the Veracity, and even others also talk about Value). This talk focused on the Velocity which can be managed by Big Data Stream Processors. These systems are operating over streams of data and can scale horizontally. The nice thing is that all of the current platforms support dynamic cluster scaling and thus a data topology gets automatically distributed. In the talk it was highlighted how primitive functions for filtering and grouping of streams according to certain fields (FieldsGrouping in Storm for instance) can be used to make sense of large data streams. The talk included application examples from this arena which mainly evolved around the future smart city and the real-time analysis of social signals.

It was particular interesting that although the talk was very late in the day still a quite good crowd attended. Thank you guys for hanging in there with me!

Download the slide-deck here

Comments are closed.