After an investigation, we adopted \ud2f0DB<\/a>, an open-source, distributed, Hybrid Transactional\/Analytical Processing (HTAP) database. Now, our production environment has 52 TiDB clusters<\/strong> that store 160+ TB of data<\/strong>, with 320+ servers<\/strong> running in 15 applications<\/strong>. To maintain such large-scale clusters, we only need two DBAs<\/strong>.<\/p>\n\n\n\n In this post, we’ll deep dive into why we migrated from MySQL to TiDB and how we use TiDB to scale out our databases, perform real-time analytics, and achieve high availability.<\/p>\n\n\n\n We stored our private cloud monitoring data in MySQL. Because our data size was huge, we needed to regularly clean our tables. This was a lot of work for our DBAs.<\/p>\n\n\n\n When we used MySQL, we encountered these problems:<\/p>\n\n\n\n Therefore, we looked for a new database solution with the following capabilities:<\/p>\n\n\n\n \ud2f0DB<\/a> is an open-source, cloud-native, distributed SQL database built by PingCAP<\/a> and its open-source community. It is MySQL compatible and features horizontal scalability, strong consistency, and high availability. It’s a one-stop solution for both Online Transactional Processing (OLTP) and OLAP workloads. You can learn more about TiDB’s architecture \uc5ec\uae30<\/a>.<\/p>\n\n\n\n We adopted TiDB, because it met all our requirements for the database:<\/p>\n\n\n\n So far, we’ve deployed 52 TiDB clusters<\/strong> in the production environment to store 160+ TB of data<\/strong>, with 320+ servers<\/strong> running in 15 applications.<\/strong> Our databases have 5.5 billion daily visits<\/strong>. To maintain such large-scale clusters, we only need two DBAs<\/strong>, and, meanwhile, they’re also managing MySQL databases.<\/p>\n\n\n\n In a TiDB cluster, our application separates read and write domain names, and the backend uses load balancers to balance read and write loads. By default, a single cluster has four TiDB nodes, one for writes and three for reads. If write performance becomes a bottleneck, we can add TiDB nodes.<\/p>\n\n\n\n The following table summarizes how we’ve used TiDB over the years and how our reliance on TiDB has grown.<\/p>\n\n\n\n We’re running 242 TiDB databases in 15 applications<\/strong>. To name a few:<\/p>\n\n\n\n Thanks to TiDB’s horizontal scalability, high availability, and HTAP capabilities, we could say goodbye to troublesome MySQL sharding and time-consuming ETL. It’s so easy to scale out our databases and maintain such large-scale clusters.<\/p>\n\n\n\n Now, at 58.com, we use databases including MySQL, Redis, MongoDB, Elasticsearch, and TiDB. We receive about 800 billion visits per day. We have 4,000+ clusters, with 1,500+ physical servers.<\/p>\n\n\n\n<\/span>Our pain points<\/span><\/h2>\n\n\n\n
\n
\n
<\/span>Why we chose TiDB, a NewSQL database<\/span><\/h2>\n\n\n\n
\n
<\/span>How we’re using TiDB as an alternative to MySQL<\/span><\/h2>\n\n\n\n
TiDB’s architecture at 58.com<\/h3>\n\n\n\n
![\"\"](\"https:\/\/www.pingcap.com\/core\/uploads\/2021\/11\/tidb-architecture-at-58.com_-1024x508.jpg\")
<\/figcaption><\/figure>\n\n\n\nFrom TiDB 2.0 to TiDB 4.0.2<\/h3>\n\n\n\n
Date<\/strong><\/td> TiDB version<\/strong><\/td> Environment<\/strong><\/td><\/tr> April 2018<\/td> TiDB 2.0 was introduced in production<\/td> \n \n
December 2018<\/td> Upgraded to TiDB 2.1<\/td> Four TiDB clusters in production storing all our private cloud’s log system data<\/td><\/tr> September 2019<\/td> Upgraded to TiDB 3.02 in production and test environments<\/td> \n \n
September 2020<\/td> Upgraded all TiDB clusters to 4.0.2<\/td> \n \n
Our applications running on TiDB<\/h3>\n\n\n\n
TiDB clusters<\/strong><\/td> Applications<\/strong><\/td> Application details<\/strong><\/td> Detailed information<\/strong><\/td> Comment<\/strong><\/td><\/tr> TEG<\/td> WList management backend and audio configuration<\/td> WList is a general-purpose distributed key-list storage platform developed by the 58 Architecture Platform Department<\/td> Data size: 6.3 billion rows of data<\/td> 7.2 TB in the past 6 months, 7 flash memory cards have been damaged in these applications.
But thanks to TiDB\u2019s high availability, they didn\u2019t affect our businesses<\/strong>.<\/td><\/tr>WTable management backend<\/td> The WTable cluster application accesses monitoring data such as machine CPU, I\/O, and network<\/td> Data size: 26.2 billion rows of data<\/td><\/tr> Search index<\/td> It stores data about user search terms used in the last year<\/td> Current data size: 300 million rows of data;
Daily increase: 500 thousand to 800 thousand rows of data<\/td><\/tr>Anjuke (a real estate information service platform)<\/td> Operation log<\/td> Operation log<\/td> Daily increase: 40 million rows of data<\/td> <\/td><\/tr> Statement log<\/td> Statement log<\/td> QPS: 10 thousand at peak
Disk: 1 TB<\/td><\/td><\/tr> User growth<\/td> 58 information<\/td> \ub274\uc2a4<\/td> Data size: 50 million rows of data
Disk: 1 TB<\/td><\/td><\/tr> Information security<\/td> Verification center<\/td> Verification center<\/td> Daily increase: 1 million rows of data<\/td> <\/td><\/tr> Financial company<\/td> The financial real-time data warehouse\u2019s underlying data storage <\/td> The financial real-time data warehouse\u2019s underlying data storage<\/td> Thousands of tables<\/td> <\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n <\/span>Conclusion<\/span><\/h2>\n\n\n\n