Tech Talk: Twitter: Storm is coming: Real-time Big Data at Twitter

Information Session: External Relations Group | January 22 | 5:30-7:30 p.m. | Soda Hall, Wozniak Lounge/430/438

 Karthik Ramasamy, Engineering Manager and Technical Lead for Real Time Analytics, Twitter; Sanjeev Kulkarni, Senior Software Engineer, Twitter

 Electrical Engineering and Computer Sciences (EECS)

Featuring Karthik Ramasamy & Sanjeev Kulkarni

Twitter is all about real time - real time conversations, real time trends, real time search and real time content dissemination. Twitter has invested in a massive data pipeline that collects, aggregates, processes large volumes of data in real time. At the heart of the pipeline is Twitter Storm a real-time stream processing engine.

In this talk, we will give an overview of real time analytics, discuss the twitter real time data pipeline and how Storm is used for extracting analytics. We will also discuss the challenges we faced and lessons we have learned while building this infrastructure at Twitter.