{"id":1463,"date":"2020-06-19T00:00:00","date_gmt":"2020-06-19T00:00:00","guid":{"rendered":"https:\/\/en.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/"},"modified":"2024-05-22T05:32:46","modified_gmt":"2024-05-22T12:32:46","slug":"vldb-2020-tidb-a-raft-based-htap-database","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/","title":{"rendered":"VLDB 2020: TiDB, A Raft-based HTAP Database"},"content":{"rendered":"

At PingCAP<\/a>, our goal is to build a database that is a single, real-time source of truth so that the applications and the businesses on top of it can adapt to uncertainty. As we built \ud2f0DB<\/a>, we benefited from databases and distributed systems research such as Raft and Spanner. Along the journey, we’ve developed some novel ideas, and we are thrilled to contribute them back to the academic community.<\/p>\n\n\n\n

A group of our engineers have summarized what we’ve learned in a new research paper, “TiDB: A Raft-based HTAP Database.” In this paper, we propose TiDB as a Hybrid Transactional and Analytical Processing (HTAP) database, a “one size fits many” solution for OLTP and OLAP applications.<\/p>\n\n\n\n

HTAP databases require processing transactional and analytical queries in isolation to remove the interference between them. To achieve this, it is necessary to maintain different replicas of data specified for the two types of queries. To this end, we propose extending replicated state machine-based consensus algorithms to provide consistent replicas for HTAP workloads. Based on this novel idea, we present a Raft-based HTAP database: TiDB.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n
A multi-Raft storage system consisting of a row store and a column store<\/div>\n\n\n\n

The figure above shows the key architecture of TiDB. TiDB is built on a multi-Raft storage system which consists of a row store (TiKV<\/a>) and a column store (TiFlash<\/a>). The row store is built based on the Raft algorithm to materialize updates from transactional requests. In particular, it asynchronously replicates Raft logs to learners which transform row format to column format for tuples, forming a column store. This column store allows analytical queries to read fresh and consistent data in real time. On the top of this storage system, we build an SQL engine to optimally access TiKV and TiFlash for HTAP queries. Comprehensive experiments show that TiDB achieves isolated high performance under CH-benCHmark, a benchmark focusing on HTAP workloads.<\/p>\n\n\n\n

TiDB presents an innovative solution that helps consensus-algorithm-based NewSQL systems evolve into HTAP systems. NewSQL systems ensure high availability, scalability, and data durability for OLTP requests by replicating their database like Google Spanner and CockroachDB. They synchronize data across data replicas via replication mechanisms typically from consensus algorithms. Based on the log replication, NewSQL systems can provide a columnar replica dedicated to OLAP requests so that they can support HTAP requests in isolation like TiDB.<\/p>\n\n\n\n

We believe that the future of the database is about unification, adaptiveness, and intelligence. When we face uncertainty in the future, we will have the unified infrastructure to adapt intelligently. We hope that “TiDB: A Raft-based HTAP Database” can inspire researchers and software engineers to make the future come earlier. The paper has been published in August, 2020, in PVLDB<\/em>, Volume 13, Issue 12. Download the paper \uc5ec\uae30<\/a>.<\/p>\n\n\n\n

We would like to offer a big “thank you” to authors and reviewers of the paper. We also want to thank the many contributors and users in our TiDB community. You have made TiDB what it is today, and you will continue to make it better tomorrow.<\/p>","protected":false},"excerpt":{"rendered":"

PingCAP will publish a research paper “TiDB: A Raft-based HTAP Database” in the August issue of the Proceedings of the VLDB Endowment.<\/p>","protected":false},"author":8,"featured_media":1465,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"ub_ctt_via":"","footnotes":""},"categories":[18],"tags":[10,67],"class_list":["post-1463","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-community","tag-htap","tag-vldb"],"acf":[],"featured_image_src":"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg","author_info":{"display_name":"TiDB Team","author_link":"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/"},"yoast_head":"\nVLDB 2020: TiDB, A Raft-based HTAP Database | TiDB<\/title>\n<meta name=\"description\" content=\"PingCAP will publish a research paper "TiDB: A Raft-based HTAP Database" in the August issue of the Proceedings of the VLDB Endowment.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"VLDB 2020: TiDB, A Raft-based HTAP Database | TiDB\" \/>\n<meta property=\"og:description\" content=\"PingCAP will publish a research paper "TiDB: A Raft-based HTAP Database" in the August issue of the Proceedings of the VLDB Endowment.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:published_time\" content=\"2020-06-19T00:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-22T12:32:46+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"2000\" \/>\n\t<meta property=\"og:image:height\" content=\"667\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TiDB Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TiDB Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\"},\"author\":{\"name\":\"TiDB Team\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74\"},\"headline\":\"VLDB 2020: TiDB, A Raft-based HTAP Database\",\"datePublished\":\"2020-06-19T00:00:00+00:00\",\"dateModified\":\"2024-05-22T12:32:46+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\"},\"wordCount\":508,\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg\",\"keywords\":[\"HTAP\",\"VLDB\"],\"articleSection\":[\"Community\"],\"inLanguage\":\"ko-KR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\",\"url\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\",\"name\":\"VLDB 2020: TiDB, A Raft-based HTAP Database | TiDB\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg\",\"datePublished\":\"2020-06-19T00:00:00+00:00\",\"dateModified\":\"2024-05-22T12:32:46+00:00\",\"description\":\"PingCAP will publish a research paper \\\"TiDB: A Raft-based HTAP Database\\\" in the August issue of the Proceedings of the VLDB Endowment.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage\",\"url\":\"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg\",\"width\":2000,\"height\":667},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"VLDB 2020: TiDB, A Raft-based HTAP Database\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74\",\"name\":\"TiDB Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"caption\":\"TiDB Team\"},\"url\":\"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"VLDB 2020: TiDB, A Raft-based HTAP Database | TiDB","description":"PingCAP will publish a research paper \"TiDB: A Raft-based HTAP Database\" in the August issue of the Proceedings of the VLDB Endowment.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pingcap.com\/ko\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/","og_locale":"ko_KR","og_type":"article","og_title":"VLDB 2020: TiDB, A Raft-based HTAP Database | TiDB","og_description":"PingCAP will publish a research paper \"TiDB: A Raft-based HTAP Database\" in the August issue of the Proceedings of the VLDB Endowment.","og_url":"https:\/\/www.pingcap.com\/ko\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_published_time":"2020-06-19T00:00:00+00:00","article_modified_time":"2024-05-22T12:32:46+00:00","og_image":[{"width":2000,"height":667,"url":"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg","type":"image\/jpeg"}],"author":"TiDB Team","twitter_card":"summary_large_image","twitter_creator":"@PingCAP","twitter_site":"@PingCAP","twitter_misc":{"Written by":"TiDB Team","Est. reading time":"3\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#article","isPartOf":{"@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/"},"author":{"name":"TiDB Team","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74"},"headline":"VLDB 2020: TiDB, A Raft-based HTAP Database","datePublished":"2020-06-19T00:00:00+00:00","dateModified":"2024-05-22T12:32:46+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/"},"wordCount":508,"publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg","keywords":["HTAP","VLDB"],"articleSection":["Community"],"inLanguage":"ko-KR"},{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/","url":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/","name":"VLDB 2020: TiDB, A Raft-based HTAP Database | TiDB","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg","datePublished":"2020-06-19T00:00:00+00:00","dateModified":"2024-05-22T12:32:46+00:00","description":"PingCAP will publish a research paper \"TiDB: A Raft-based HTAP Database\" in the August issue of the Proceedings of the VLDB Endowment.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/"]}]},{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#primaryimage","url":"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg","width":2000,"height":667},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"VLDB 2020: TiDB, A Raft-based HTAP Database"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]},{"@type":"Person","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74","name":"TiDB Team","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/","url":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","caption":"TiDB Team"},"url":"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/"}]}},"grav_blocks":false,"card_markup":"<a class=\"card-resource bg-white\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/vldb-2020-tidb-a-raft-based-htap-database\/\"><div class=\"card-resource__image-container\"><img class=\"card-resource__image\" alt=\"a-raft-based-htap-database.jpg\" src=\"https:\/\/static.pingcap.com\/files\/2020\/06\/a-raft-based-htap-database.jpg\" loading=\"lazy\" width=2000 height=667 \/><\/div><div class=\"card-resource__content-container\"><div class=\"card-resource__content-head\"><div class=\"card-resource__category\">Community<\/div><\/div><h5 class=\"card-resource__title\">VLDB 2020: TiDB, A Raft-based HTAP Database<\/h5><\/div><\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/1463","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/comments?post=1463"}],"version-history":[{"count":5,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/1463\/revisions"}],"predecessor-version":[{"id":17065,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/1463\/revisions\/17065"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media\/1465"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=1463"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/categories?post=1463"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/tags?post=1463"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}