{"id":8698,"date":"2022-08-30T02:14:24","date_gmt":"2022-08-30T09:14:24","guid":{"rendered":"https:\/\/en.pingcap.com\/?p=8698"},"modified":"2025-11-14T06:48:38","modified_gmt":"2025-11-14T14:48:38","slug":"how-good-is-tidb-as-an-htap-system-a-hattrick-benchmark","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/how-good-is-tidb-as-an-htap-system-a-hattrick-benchmark\/","title":{"rendered":"How Good is TiDB as an HTAP System? A HATtrick Benchmark"},"content":{"rendered":"

I\u2019m one of the developers of TiDB, a Hybrid Transactional and Analytical (HTAP) database. Recently, I read the SIGMOD 22 paper, \u201cHow Good is My HTAP System<\/a>,\u201d which uses TiDB as one of its research subjects. I really appreciate the authors\u2019 work \u2014, especially their methodology and the tooling they created to observe and benchmark HTAP performance. For example, the HATtrick benchmark tool not only produces Online Transactional Processing (OLTP) and Online Analytical Processing (OLAP) workloads simultaneously, it also evaluates the freshness of OLAP queries on an HTAP system. <\/p>\n\n\n\n

The paper concludes that analytical queries running on TiDB always read the latest data. This is an essential feature for businesses that want to make decisions based on the most current information.<\/p>\n\n\n\n

However, the TiDB performance test results weren\u2019t up to our standards. In an effort to learn more, I reproduced the HATtrick tests and discussed them here. I hope this information gives you greater insight to HTAP in general and TiDB specifically. <\/p>\n\n\n\n

<\/span>How HATtrick works<\/strong><\/span><\/h2>\n\n\n\n

The author\u2019s benchmark tool, HATtrick, combines the Star Schema Benchmark (SSB) for analytical workloads and the TPCC benchmark for transactional workloads. This design allows HATtrick to send transactional queries and analytical queries to the database simultaneously to measure the performance isolation between the two workloads. The paper also introduces the concept of \u201cthroughput frontier,\u201d which shows transactional throughput and analytical throughput in one diagram. Throughput frontier is an intuitive way to track performance isolation between transactional and analytical processing (AP) and (TP).<\/p>\n\n\n\n

For example, in the following frontier diagram, the red dotted line is the \u201cbounding box,\u201d and the blue dotted line is the \u201cproportional line.\u201d The green solid line indicates the system’s capability when it runs AP and TP queries simultaneously. The closer the green solid line is to the bounding box, the better performance isolation between TP and AP.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Performance isolation shown in the frontier<\/em><\/p>\n\n\n\n

What is HTAP \u201cfreshness\u201d?<\/strong><\/h3>\n\n\n\n

This paper also defined the \u201cfreshness\u201d of an HTAP system: <\/p>\n\n\n\n

\u00a0An HTAP system provides fresh analytics if every analytical query is executed on an up-to-date version of the operational data<\/a>. Else, it provides stale analytics.\u00a0<\/em><\/p>\n\n\n\n

HATtrick benchmark includes a freshness table that records the version of the transactional queries that have been processed. The benchmark uses this table to check the versions retrieved by the analytical queries. If the analytical queries always retrieve the latest version of the data, the freshness score is 0s. In the following frontier chart, the freshness score is 1.5s; that is, the analytical queries can only retrieve the data that\u2019s been stale for 1.5 seconds.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

A frontier chart freshness score<\/em><\/p>\n\n\n\n

<\/span>Run HATrick on TiDB again<\/strong><\/span><\/h2>\n\n\n\n

TiDB is a distributed HTAP database. It has a dedicated row engine, TiKV, and a dedicated column engine, TiFlash. The TiDB optimizer automatically routes transactional queries to TiKV and analytical queries to TiFlash. Theoretically, if there are no resource conflicts between TiDB, TiKV, and TiFlash, there is great performance isolation between the two types of queries. <\/p>\n\n\n\n

Also, TiDB has a disaggregated architecture that separates storage and computing. This allows users to scale AP or TP throughput by adding or removing TiDB instances as needed.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

TiDB architecture<\/em><\/p>\n\n\n\n

My retesting with HATtrick focused on: <\/p>\n\n\n\n