Why Databricks Migrated to a Scalable Data Architecture

Overview

Databricks was using MySQL to manage its control plane containing multiple cloud services supporting users, cluster management, and web applications. However, as the company’s cloud usage grew by 3x, MySQL was unable to handle the increased load: queries became slow or even unresponsive for large customers.

In this customer story video, Databricks Staff Software Engineer Min Zhou sheds light on how the company’s migration to TiDB—an advanced open-source, distributed SQL database—lowered hardware costs while increasing QPS performance by 10x. He shares the challenges Databricks had with its original solution, its criteria for evaluating a new database solution, and how TiDB helps the company achieve its goals in production.