Title: To unify or to specialize?

Abstract: When designing a new system one key decision we need to make is what kind of workloads the system should support. Some systems focus on handling one workload particularly well (e.g., Graph databases, streaming), while others target a broader range of workloads (e.g., Apache Spark). However, whether a system is specialized or provides unified support for multiple workloads is not always clear cut. Over their lifetime, some systems evolve to handle an increasingly diverse set of workloads. In this talk, I will share our experience with building several such systems over the past decade (e.g., Apache Mesos, Apache Spark, Ray), describe the evolution of these systems in the context of the broader ecosystem, and speculate on what could make unified or specialized systems successful.

Bio: Ion Stoica is a Professor in the EECS Department at University of California at Berkeley, and the Director of RISELab. He is currently doing research on cloud computing and AI systems. Past work includes Apache Spark, Apache Mesos, Tachyon, Chord DHT, and Dynamic Packet State (DPS). He is an ACM Fellow and has received numerous awards, including the SIGOPS Hall of Fame Award (2015), the SIGCOMM Test of Time Award (2011), and the ACM doctoral dissertation award (2001). He is Executive Chairman at Databricks, a company he co-founded in 2013 to commercialize Apache Spark. In 2006 he also co-founded Conviva, a startup to commercialize technologies for large scale video distribution.