Twitter is all about real time - real time conversations, real time trends, real time search and real time content dissemination. Twitter has invested in a massive data pipeline that collects, aggregates, processes large volumes of data in real time. At the heart of the pipeline is Twitter Storm a real-time stream processing engine.
In this talk, we will give an overview of real time analytics, discuss the twitter real time data pipeline and how Storm is used for extracting analytics. We will also discuss the challenges we faced and lessons we have learned while building this infrastructure at Twitter.