© 2020 Strange Loop
The need for gleaning answers from data in real-time is moving from nicety to a necessity. There are few options to analyze the never-ending stream of unbounded data at scale. Let's compare and contrast the core principles and technologies the different open source solutions available to help with this endeavor, and where in the future processing engines need to evolve to solve processing needs at scale.
These findings are based on of our experience of building and continuing to build a scalable solution in the cloud to process over 700 billion events at Netflix, and how we have embark on the next journey to evolve unbounded data processing engines.
Monal Daxini is a Senior Software Engineer at Netflix building a scalable and multi-tenant event processing pipeline. He has worked on Netflix's Cassandra & Dynamite infrastructure, and was instrumental in developing the encoding compute infrastructure for all Netflix content. He has 15 years of experience building scalable distributed systems at organizations like Netflix, NFL.com, and Cisco.