Revving Up Efficiency: Transforming NIO’s Data Strategy with TiDB

NIO, a pioneer in the electric vehicle industry, has always embraced cutting-edge technologies to stay ahead in the highly competitive automotive market. As NIO’s business expanded, so did the complexity of its data management requirements. The company faced significant challenges as its MySQL-based database struggled to handle tables exceeding 2 billion records, multi-table joins, and increasingly complex queries. The growing strain on performance and scalability threatened to hinder NIO’s ability to maintain operational efficiency and deliver a seamless customer experience.

Recognizing the need for a more robust and scalable solution, NIO turned to TiDB—a distributed SQL database known for its high availability, horizontal scalability, and compatibility with MySQL. This case study details NIO’s migration journey from MySQL to TiDB, illustrating how the implementation resolved key data challenges, improved system performance, and enhanced overall business operations.

About NIO

NIO is a global leader in smart electric vehicles, dedicated to delivering high-performance EVs coupled with an exceptional user experience. In Q3 2023, China’s automotive market reached sales of 5.668 million units, reflecting a 2.4% year-on-year growth. Among these, new energy vehicle sales surged by nearly 2 million units, marking an impressive 36% year-on-year increase. NIO emerged as the dominant player in China’s premium electric vehicle segment (priced above 300,000 RMB), securing a remarkable 45% market share, solidifying its leadership in the fast-growing new energy vehicle sector.

The Challenges: The Growing Pains of Managing Massive Data Volumes

As NIO’s business expanded rapidly, the company encountered significant challenges in managing the growing volume of data across various business lines. Some internal systems were generating tens of millions of new records daily, causing the size of certain tables in their MySQL databases to exceed 2 billion records. This massive data growth led to the following key issues:

• Performance Bottlenecks: Many business scenarios required complex queries that involved joining large tables. Some of these queries spanned tables containing billions of records, leading to severe performance slowdowns. Aggregate functions such as COUNT suffered the most, often resulting in query inefficiencies and frequent timeouts.
• Inefficiencies in Time-Range Queries: Most queries needed to filter data over the last six months, while also accessing historical data. This led to performance drops when dealing with large periods.
• Complex Table Structures: NIO’s large tables contained over 30 fields, with around 10 fields frequently used in joins with other tables. This complexity further strained the system’s query performance, especially when combined with massive record sizes.
• Write and Replication Delays: Some of the tables handled immense data writes, which caused significant delays in master-slave replication processes, directly affecting the performance of multiple business operations.
• Slow DDL Execution: Due to the enormous size of these tables, executing Data Definition Language (DDL) operations in MySQL became extremely slow, sometimes taking hours to complete.

NIO’s original data architecture, based on MySQL with traditional sharding strategies, was no longer sufficient to meet the growing demands of their business. 

Original Data Architecture 

NIO’s original data architecture was based on traditional relational databases that handle typical enterprise data workloads. The architecture was distributed across multiple data centers, relying on batch processing and manual interventions to synchronize and manage data flow. Key characteristics of this architecture included:

• Monolithic Database Systems: NIO relied heavily on a monolithic relational database system, which worked well during the early stages of its growth but became a bottleneck as the volume and velocity of data increased. This setup struggled with high-frequency data updates, leading to latency and limited scalability.
• Batch Processing: To manage the enormous influx of data, the system relied on batch processing, which meant data was processed in chunks at intervals. This method could not meet the real-time analytics demands of NIO’s business, especially for in-vehicle telemetry and customer interaction data, which required immediate insights.
• Manual Sharding and Data Partitioning: Because the relational database could not scale automatically, NIO had to implement manual sharding and data partitioning. This process complicated the architecture and made maintenance cumbersome and error-prone.
• Data Silos and Inconsistencies: With multiple data centers and databases in place, maintaining data consistency across regions became a challenge. This resulted in data silos, making it difficult for NIO to achieve a unified view of critical business and vehicle-related data.

This architecture, while initially sufficient, became increasingly strained under the pressure of NIO’s rapid growth, necessitating a shift to a more modern, scalable, and real-time data solution like TiDB.

The Solution: Transitioning to TiDB for Scalability and Performance

Why did NIO Choose TiDB? 

After research, the NIO data team explored distributed databases, focusing on TiDB, a widely recognized open-source distributed HTAP (Hybrid Transactional/Analytical Processing) database.

Through their research, the team found TiDB to be a distributed relational database suitable for online transactions and analytical processing. It features horizontal scalability, financial-grade high availability, real-time HTAP, cloud-native capabilities, MySQL 5.7/MySQL 8.0 protocol compatibility, and MySQL ecosystem support. It provides a one-stop solution for OLTP, OLAP, and HTAP scenarios, making it ideal for high availability, strong consistency, and large-scale data environments.

TiDB’s numerous advantages met NIO’s data team’s needs for handling large-scale data and high-concurrency transactions:

1. Distributed Architecture: TiDB uses a distributed relational database architecture, which overcomes single-machine processing limitations and enhances overall performance and scalability.
2. High Availability: TiDB employs the Raft consensus algorithm, replicating data across TiKV nodes for fault tolerance and ensuring data security in case of node failures. It supports dual data centers and two-site, three-center high-availability solutions.
3. Horizontal Scalability: TiDB supports the functionalities of traditional relational databases while providing the flexibility and horizontal scalability of non-relational databases.
4. Distributed ACID Transactions: TiDB maintains ACID properties, ensuring data consistency and integrity in distributed environments.
5. MySQL Compatibility: TiDB is highly compatible with MySQL protocols, SQL syntax, and ecosystem tools, reducing migration costs and allowing a smooth transition from MySQL.
6. Flexible Partitioning: TiDB supports various partitioning schemes (hash, range, list, key), simplifying data management and improving query efficiency.
7. Powerful Data Synchronization Tools:
   • DM (Data Migration): Facilitates full and incremental data synchronization from MySQL to TiDB.
   • TiCDC: Enables data synchronization based on binlog, allowing master-slave replication between TiDB and MySQL or TiDB-to-TiDB setups.
8. Rich Ecosystem: TiDB’s ecosystem includes TiFlash for columnar storage to optimize analytical queries and TiSpark for integrating TiDB as a storage layer with Spark for robust big data processing.

These features provided NIO with a high-performance and high-availability database solution and supported its data management and analysis needs, driving ongoing business innovation.

Implementation Strategy

1. Assessment of Needs: NIO began by conducting a comprehensive assessment of its existing data architecture, identifying critical performance bottlenecks and understanding the specific requirements of various business scenarios. This involved mapping out data workflows, query patterns, and the limitations of the existing MySQL setup.
2. Data Migration: NIO’s data team established a controlled and reliable migration process. Initially, all user business data continued to be written into the MySQL database to maintain stability. Using DM (TiDB Data Migration), the team synchronized this data into TiDB, ensuring data integrity during the transition. Internal systems then began querying TiDB directly, which led to significant optimizations in query performance.
3. Performance Validation: Following the initial data synchronization, the NIO data team conducted extensive validation. They used a single table containing 2 billion rows of business data for performance comparison between MySQL and TiDB. This validation was critical to ensure that TiDB met or exceeded the performance benchmarks set by the previous system.
4. Optimizing Storage Architecture: To further enhance performance, NIO leveraged TiDB’s unique storage architecture. The TiKV distributed key-value storage layer, based on RocksDB, optimized write performance and allowed for better disk space utilization. The team implemented partition tables to narrow down potential query ranges, significantly improving the efficiency of data access. Additionally, they utilized TiFlash’s columnar storage capabilities, enhancing query efficiency for analytical workloads.
5. Leveraging MVCC: TiDB’s Multi-Version Concurrency Control (MVCC) mechanism allowed NIO to handle high-concurrency read-write operations effectively. By assigning a globally unique timestamp to each transaction, TiDB facilitated consistent data snapshots across multiple transactions, which improved concurrent performance.
6. Ongoing Monitoring and Optimization: NIO established a robust monitoring framework to track system performance and user feedback post-implementation. This ongoing assessment enabled the team to make real-time adjustments, ensuring that TiDB continuously met the evolving needs of the business.

The Results: Driving Efficiency in NIO’s Data Ecosystem

The implementation of TiDB significantly transformed NIO’s data management landscape. Key outcomes included:

• Enhanced Query Performance: After the optimization efforts, TiDB’s join queries achieved remarkable performance metrics, with 80% of queries executing within 2 seconds and the remaining 20% within 5 seconds. The rapid execution of count results further improved user satisfaction.
• Increased Scalability: TiDB’s distributed architecture provided the scalability needed to handle growing data volumes without compromising performance, allowing NIO to efficiently manage tens of millions of records added daily.
• Operational Efficiency: Improved data access and query efficiency positively impacted overall business operations, enabling faster decision-making and better user experiences.
• Stability and Confidence: NIO has widely adopted TiDB across multiple business units. Team feedback indicated that TiDB effectively solved many business challenges and demonstrated exceptional stability, significantly boosting confidence in the use of domestic distributed databases.

Conclusion 

The successful implementation of TiDB at NIO has marked a significant turning point in the company’s data management strategy, addressing critical challenges posed by massive data volumes and complex queries. By transitioning from a traditional MySQL database to a distributed architecture with TiDB, NIO has enhanced query performance and efficiency and improved overall business processes and user experiences. The optimization of data retrieval times, particularly in join queries, demonstrates TiDB’s capability to support high-demand scenarios typical of the fast-paced electric vehicle market.

As NIO continues to expand and innovate, the ongoing adoption of TiDB across its business units solidifies the foundation for future growth and success. This transformation is just the beginning, setting a benchmark for how modern data solutions can propel organizations into new realms of efficiency and customer satisfaction.

Join NIO in experiencing the transformational impact of TiDB—discover how you can optimize your data strategy.