How a NewSQL Database Helps Keep Data Fresh and Maximize Its Business Value

Author: Jun Yu (Solution Architect at PingCAP)

Transcreator: Fendy Feng; Editors: Tom Dewan, Caitin Chen

In today’s digital economy, transactions generate various types of data. For businesses, this creates some challenges. How do you ensure that you’re making decisions based on the freshest data available and the most updated intelligence derived from that data? How do you process massive amounts of data as efficiently as possible without deficiencies or data “silos”? How do you maximize the business value of that data? 

In this post, I want to talk about the pain points of current data solutions and share how TiDB, a highly scalable enterprise-level distributed database, helps users keep their data fresh and translate their data into business value. 

Changing demands in data processing

As the world is going deep in the digital transformation, the demands for data technologies are changing and more challenging. Today, businesses need to:

• Rapidly process tremendous amounts of data in real time to keep its fresh value
• Make all links of the business chain digital and data-driven
• Rapidly derive insights from data in real time and translate these insights into commercial value

The flow chart of data processing

To meet these demands, companies need to upgrade their technologies—especially data processing technologies. 

Pain points in current data technologies

Data silos

Sometimes, a company’s data infrastructure gets out of sync. The way they use and manage data has changed a lot, but their data processing technologies may have stayed the same for 40 or 50 years. This mismatch leaves them few choices but to combine different technologies in different stacks to handle the data. Data in different stacks, however, is barely accessible, and creates  “data silos.”

Failure to keep data fresh

To bring together data from different stacks, a variety of data tools have been introduced, such as data synchronization tools with longer pipelines. But this can create new problems. Although data silos are connected by using data tools, data becomes out of date after going through a long and complex processing process. Real-time data cannot stay fresh. 

Weak commercial value

After massive fresh data is generated, businesses usually want to derive insights from this data for different commercial purposes. To meet this end, you have to process data a second time using complicated data stack complexes, which once again adds to the length of the data processing course. This inevitably leads to a loss of data’s real-time value and weaker commercial value. 

Request a Demo     Subscribe to Blog

How TiDB helps different industries achieve business impact with data

Now that we know the demands for data processing and the downsides of current data solutions, let’s see TiDB’s strengths, and most importantly, how it helps various businesses keep their data fresh and achieve their data’s commercial value. 

What is TiDB?

TiDB is a horizontally scalable, strongly consistent, and highly available open-source NewSQL database. It gives users a one-stop solution with Online Transactional Processing (OLTP), Online Analytical Processing (OLAP), and real-time Hybrid Transactional and Analytical Processing (HTAP) services. 

The scalable OLTP capability of TiDB helps business users rapidly process massive amounts of incoming data. This keeps the data as fresh as possible. In addition, TiDB’s real-time HTAP ability can provide users with fresh intelligence related to risk control, marketing, and other areas for commercial decision-making, while not interfering with the OLTP tasks.

TiDB’s data processing flow

Industry use cases

TiDB has been adopted by thousands of businesses of various shapes and sizes in many industries including logistics, internet, and financial services. Let’s see how TiDB helps them upgrade their data infrastructure and achieve business impact with data. 

Industry: financial services

• Customer: Yillion Bank, a leading internet bank in China with the majority of its business coming from the internet 
  
  How does TiDB help?
  
  TiDB helps build Yillion Bank’s distributed database system to:
  • Support its mission-critical banking systems, loan and mortgage services, and asset-backed securitization business
  • Support multi-data center disaster recovery in Beijing and Changchun, two big cities in China 
  • Migrate heterogeneous data seamlessly from Oracle to TiDB
  • Ensure that if the number of transactions greatly increase, the system will stay resilient

• Customer: Pingan Life Insurance, a subsidiary of Pingan Insurance Group which is the world-leading technology-powered retail financial services company
  
  How does TiDB help?
  
  The major characteristic of Pingan’s agile financial systems—and also the major challenge of its data infrastructure—is peak transactions occurring in a short time. During the annual “Wealth God Festival” campaign on Pingan’s Gold Keeper mobile app, the transaction volume once reached a record 100 billion yuan on a single day. 
  
  Behind these peak transactions, hundreds of TiDB database instances were unremittingly delivering smooth frontend queries and convenient backend operations. To be more specific, TiDB: 
  
  • Uses standard SQL syntax to speed up agile application development
  • Scales out and in quickly by adding or removing nodes online without downtime
  • Uses secondary indexes and SQL statements to analyze real-time data
  • Simplifies database operations

• Customer: a leading mobile payment company in Japan and with more than 29 million individual users out of Japan’s 100 million population and over 2 million business users
  
  How does TiDB help?
  
  Previously, our customer used the Amazon Aurora database. However, they had problems scaling it out during peak payment transactions. TiDB, which features horizontal scalability and high throughput, has helped our customer:
  
  • Seamlessly migrate its data from Amazon Aurora to TiDB clusters
  • Quickly scale out or in during large transactions 

• Customer:Bank of Beijing (BOB), a leading city commercial bank in China
  
  How does TiDB help?
  
  We have helped BOB build its comprehensive real-time data service platform. This platform:
  
  • Decouples many modern applications from BOB’s legacy systems such as AS400 and DB2; TiDB can replicate all the real-time data from those systems, including up to 10 billion rows of payment transaction data from the mission-critical banking system, and respond to real-time online queries
  • Consolidates data from various sources into TiDB to provide comprehensive data services 
  • Upholds diverse financial scenarios including payment, accounting, and financial channels with the help of TiDB’s hybrid partitioning and real-time HTAP capabilities
  • Makes a ZERO error record when supporting BOB’s mission-critical financial systems including the payment and credit systems

• Customer:Beike Finance, China’s largest provider of residential property financing services
  
  How does TiDB help?
  
  We have helped Beike build its real-time data center so that it:
  
  • Integrates TiDB with stream processing systems such as Apache Flink and Apache Kafka
  • Forms a comprehensive system to support real-time data generation, processing, and synchronization

Industry: catering

• Customer: A world-leading catering giant
  
  How does TiDB help?
  
  TiDB helps our customer: 
  
  • Build its mission-critical database to support its highly concurrent and scalable transaction systems including its order center, user center, and coupon center
  • Integrate with Kubernetes (K8s) to adapt to our client’s production environment by leveraging TiDB’s strengths in cloud-native architecture
  • Ensure safe and stable transactions 24×7 by leveraging TiDB’s strengths in availability, cloud-native architecture, and multi-datacenter disaster recovery 

Industry: logistics

• Customer: ZTO Express, one of the world’s largest express delivery 
  
  How does TiDB help?
  
  TiDB helped ZTO Express increase their IT efficiency by 300%, and reduce their cost per order by almost 20% year-on-year. An important step was upgrading their data system architecture with TiFlash, TiDB’s analytical engine, and TiCDC, a tool to replicate TiDB’s incremental data, to power their HTAP scenarios. 
  
  We also help ZTO Express develop their real-time analytics platform which:
  
  • Uses TiSpark to aggregate data from multiple sources and writes data into TiDB in real time
  • Can process 300 million rows of delivery data and whose queries per second (QPS) can reach 500,000/s

Try us out!

We feel proud that TiDB, our flagship product, is gaining recognition from users in diverse industries. We owe our success to the unremitting efforts of all PingCAP engineers and community contributors in the past six years. 

With horizontal scalability, strong availability, and OLTP and real-time HTAP capabilities, TiDB has become a key data infrastructure and has helped so many businesses process their data and empower the data to realize customers’ business value. 

If you want to learn more about TiDB, you’re welcome to follow TiDB on GitHub or join our community discussions on Slack. You can also request a demo from us.