{"id":18740,"date":"2024-08-08T01:37:07","date_gmt":"2024-08-08T08:37:07","guid":{"rendered":"https:\/\/www.pingcap.com\/?post_type=case-study&p=18740"},"modified":"2024-08-08T01:37:14","modified_gmt":"2024-08-08T08:37:14","slug":"trip-com-boosts-real-time-data-processing-and-financial-settlement-with-tidb","status":"publish","type":"case-study","link":"https:\/\/www.pingcap.com\/ko\/case-study\/trip-com-boosts-real-time-data-processing-and-financial-settlement-with-tidb\/","title":{"rendered":"Enhancing Real-Time Data Processing and Financial Settlement at Trip.com with TiDB"},"content":{"rendered":"

Trip.com is a leading global one-stop travel platform, encompassing brands such as Trip.com, Qunar, and Skyscanner. Trip.com offers services to over 90 million members, including hotel reservations, reviews, special hotel searches, flight bookings, status and schedule inquiries, ticket price searches, and flight queries.<\/p>\n\n\n\n

Use Case 1: Speed Up the 1-billion-data Real-time Customer Data Platform<\/h2>\n\n\n\n

Scenario Description<\/h3>\n\n\n\n

In the international business realm, Trip.com operates in a diverse market with complex products, multiple distribution channels, and high customer acquisition costs. This complexity necessitates detailed management and optimization of business operations and products to meet market needs, reduce costs, and improve efficiency and conversion rates. To address these challenges, Trip.com developed the International Business Dynamic Real-Time Tagging Processing Platform (CDP).<\/p>\n\n\n\n

Trip.com’s data is characterized by diverse sources and formats, and a mix of offline and online processing. The challenge is to collect, manage, and process this data to produce tags that meet business, operational, and market needs. The processed data must be immediately applicable to business systems, demanding high timeliness, accuracy, stability, and flexibility.<\/p>\n\n\n\n

To solve these issues, the CDP system must enhance its data processing capabilities. Previously, traditional solutions involved T+1 calculations through a data warehouse, with data then imported into ES clusters. Queries were assembled based on conditions, leading to slower response times due to offline computation and reliance on ES indexes. With over a hundred tags deployed and more than 50% used for querying, this offline approach led to poor data timeliness.<\/p>\n\n\n\n

Solution<\/h3>\n\n\n\n

The CDP aims to improve data processing timeliness while meeting business flexibility requirements. Data processing and update logic can be dynamically configured through system rules to process messages (Kafka or QMQ) and update tags. The business layer focuses only on data filtering logic and conditional queries.<\/p>\n\n\n\n

Based on business needs, data tagging is divided into two main scenarios:<\/p>\n\n\n\n

    \n
  1. Real-Time Trigger Scenario<\/strong>: Dynamic rules are configured to subscribe to changes in business systems in real-time. Data meeting dynamic rule conditions is filtered and pushed to downstream business entities.<\/li>\n\n\n\n
  2. Tag Persistence Scenario<\/strong>: Real-time business change messages are processed into business-related feature data and persistently stored in a storage engine. Queries are assembled based on business needs to access data from OLAP (analytical) and OLTP (transactional) engines.<\/li>\n<\/ol>\n\n\n\n

    The CDP system uses a Kappa-like architecture for streaming data and a Lambda-like architecture for tag persistence. <\/p>\n\n\n\n

    \"CDP<\/figure>\n\n\n\n

    \ud2f0DB<\/a> is utilized for storing persistent business tags and dynamically configured to handle real-time data changes, ensuring the timeliness of business tags. TiDB supports both OLTP and OLAP query scenarios, addressing different business needs.<\/p>\n\n\n\n

    The system adopts Lambda data processing architecture concepts, with new data routed to different channels based on source. Historical full data is batch-processed (e.g., using Spark) and written to TiDB. Incremental data is sent as messages to Kafka or QMQ queues, processed according to tagging persistence rules, and incrementally written to TiDB, addressing timeliness issues.<\/p>\n\n\n\n

    TiDB offers two main storage methods: TiKV<\/a> (row storage) for OLTP scenarios and TiFlash<\/a> (column storage) for OLAP scenarios. TiDB\u2019s internal mechanisms handle synchronization between these engines, allowing clients to choose query methods based on their needs. TiDB also ensures strong consistency and isolation, effectively addressing isolation and column storage synchronization issues in HTAP<\/a> scenarios.<\/p>\n\n\n\n

    Currently, the CDP has been deeply integrated with Trip.com\u2019s various business systems, providing data and services to support international business growth.<\/p>\n\n\n\n

    Application Value<\/h3>\n\n\n\n