{"id":5988,"date":"2022-04-12T01:43:20","date_gmt":"2022-04-12T08:43:20","guid":{"rendered":"https:\/\/en.pingcap.com\/?p=5988"},"modified":"2024-12-20T04:04:50","modified_gmt":"2024-12-20T12:04:50","slug":"how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/","title":{"rendered":"How to Achieve High-Performance Data Ingestion to TiDB in Apache Flink"},"content":{"rendered":"\n<p><a href=\"https:\/\/www.pingcap.com\/tidb\/\">TiDB<\/a> is a distributed SQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads. Apache Flink is the most popular, open-source computing framework. It provides high-throughput, low-latency data computing and exactly-once semantics.<\/p>\n\n\n\n<p>If you use TiDB with Flink, you may have had to quickly ingest streaming data from Apache Flink to TiDB. However, if you use Apache Flink\u2019s default configuration, data ingestion performance will be limited. To achieve higher performance, you must:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Enable bulk insert.<\/li>\n\n\n\n<li>Establish a high concurrency connection.<\/li>\n\n\n\n<li>Run the SQL <code>ON DUPLICATE KEY UPDATE<\/code> clause.<\/li>\n<\/ul>\n\n\n\n<p>In this article, we share two ways you can achieve high-performance data ingestion by adjusting some parameters and code.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Test_environment\"><\/span><strong>Test environment<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Our test environment was:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Apache Flink: V1.13.5<\/li>\n\n\n\n<li>Programing language: Scala V2.12<\/li>\n\n\n\n<li>MySQL JDBC Connector: V8.0.27<\/li>\n\n\n\n<li>Database: TiDB V5.3.0<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Sample_test_cases\"><\/span><strong>Sample test cases&nbsp;<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>There are different approaches to achieving high performance in this scenario.&nbsp; We will recommend two of them, both of which enable BULK INSERT. The sample code is written in Scala.&nbsp;&nbsp;&nbsp;<\/p>\n\n\n\n<p><strong>Solution 1: Use the Flink SQL statement<\/strong><\/p>\n\n\n\n<p>The Flink SQL statement is the easiest way to enable BULK INSERT from other data sources to TiDB.&nbsp;<\/p>\n\n\n<ol>\n<li>Create the SQL table:\n<pre><code>var createTableStatement = \"\"\"<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| CREATE TABLE order_item (<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| id BIGINT,<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| productId INT,<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| name STRING,<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| price DECIMAL(8,2),<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| cnt INT,<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| PRIMARY KEY (`id`) NOT ENFORCED<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| ) WITH (<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'connector' = 'jdbc',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'url' = 'jdbc:mysql:\/\/&lt;tidb_addr&gt;:4000\/?useServerPrepStmts=true<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| &amp;cachePrepStmts=true&amp;rewriteBatchedStatements=true',&nbsp;<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'driver' = 'com.mysql.jdbc.Driver',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'table-name' = 't_order_item',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'username' = 'root',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'password' = '',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'sink.buffer-flush.max-rows' = '200',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'sink.buffer-flush.interval' = '3',<\/code><br><code>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;| 'sink.parallelism' = '200' )&nbsp;<\/code><br><code> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;|\"\"\".stripMargin<\/code><\/pre>\n<p>The SQL statement above creates a table in Apache Flink that maps to the TiDB table. For better performance, note the following:<\/p>\n<table style=\"border-collapse: collapse; width: 100%; height: 140px;\">\n<tbody>\n<tr style=\"height: 35px;\">\n<td style=\"width: 39.4737%; height: 35px;\">Parameter<\/td>\n<td style=\"width: 60.5263%; height: 35px;\">Comment<\/td>\n<\/tr>\n<tr style=\"height: 35px;\">\n<td style=\"width: 39.4737%; height: 35px;\"><code>rewriteBatchedStatements<\/code><\/td>\n<td style=\"width: 60.5263%; height: 35px;\">Rewrite single statement to batch; set to true<\/td>\n<\/tr>\n<tr style=\"height: 35px;\">\n<td style=\"width: 39.4737%; height: 35px;\"><code>sink.buffer-flush.max-rows<\/code><\/td>\n<td style=\"width: 60.5263%; height: 35px;\">Batch size of a bulk insert<\/td>\n<\/tr>\n<tr style=\"height: 35px;\">\n<td style=\"width: 39.4737%; height: 35px;\"><code>sink.parallelism<\/code><\/td>\n<td style=\"width: 60.5263%; height: 35px;\">Number of concurrent connections to TiDB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/li>\n<li>Execute the insert SQL statement.\n<p>You can execute the following code to insert data to TiDB. The first line creates a table using the SQL statement from step 1.<\/p>\n<p>The second line is the data from a data source table named <code>mock_orderitem<\/code>. When the code runs, it generates the insert SQL statement. For example: <code>insert into t (a) values (10), (11), (12) on duplicate key update a = values(a);<\/code><\/p>\n<pre><code>tableEnv.executeSql(createTableStatement); \ntableEnv.executeSql(\"\"\"INSERT INTO order_item (\n      | id, productId, name, price, cnt)  \n      | select id, productId, name, price, cnt \n      | FROM mock_orderitem\"\"\".stripMargin)\n<\/code><\/pre>\n<\/li>\n<\/ol>\n\n\n<p><strong>Solution 2: Use JDBC sink in your code<\/strong><\/p>\n\n\n\n<p>Developers favor this solution because it\u2019s highly flexible. You can customize the data source, structure, and format. The sample code shown below:&nbsp;<\/p>\n\n\n<ol>\n<li>Sets TiDB connection parameters.<\/li>\n<li>Uses \u201cJdbcStatementBuilder\u201d to generate SQL statements dynamically.<\/li>\n<li>Executes the SQL statement with multi-threads.<\/li>\n<\/ol>\n<pre><code>val executionOptions = JdbcExecutionOptions.builder.withBatchIntervalMs(3).withBatchSize(200).build <\/code><br><code>val connectionOptions = (new JdbcConnectionOptions.JdbcConnectionOptionsBuilder) .withUrl(\"jdbc:mysql:\/\/&lt;tidb_addr&gt;:4000\/?useServerPrepStmts=true&amp;cachePrepStmts=true&amp;rewriteBatchedStatements=true\")<\/code><br><code>.withDriverName(\"com.mysql.jdbc.Driver\").withUsername(\"root\").withPassword(\"\").build <\/code><br><code>var insertSQL = \"INSERT INTO t_order_item (id, productId, name, price, cnt) values (?, ?, ?, ?, ?) ON DUPLICATE KEY UPDATE id=values(id)\"<\/code><br><code>val sb: JdbcStatementBuilder[OrderItem] = new JdbcStatementBuilder[OrderItem] { <\/code><br><code>       override def accept(ps: PreparedStatement, t: OrderItem): Unit = {<\/code><br><code>         ps.setLong(1, t.id); <\/code><br><code>         ps.setInt(2, t.productId); <\/code><br><code>         ps.setString(3, t.name); <\/code><br><code>         ps.setInt(4, t.price); <\/code><br><code>         ps.setInt(5, t.cnt); <\/code><br><code>    } <\/code><br><code>}<\/code><br><code>val mySink = JdbcSink.sink(insertSQL, sb, executionOptions, connectionOptions) sourceDataStream.addSink(mySink).setParallelism(200) env.execute(\"TiDB Bulk Insert Job\")<\/code><\/pre>\n\n\n<p> Note the following parameters in your code:<\/p>\n\n\n<table style=\"border-collapse: collapse; width: 100%; height: 111px;\">\n<tbody>\n<tr style=\"height: 26px;\">\n<td style=\"width: 50%; height: 26px;\">Parameter<\/td>\n<td style=\"width: 50%; height: 26px;\">Comment<\/td>\n<\/tr>\n<tr style=\"height: 26px;\">\n<td style=\"width: 50%; height: 26px;\"><code>withBatchIntervalMs(ms)IU<\/code><\/td>\n<td style=\"width: 50%; height: 26px;\">Interval for bulk insert<\/td>\n<\/tr>\n<tr style=\"height: 26px;\">\n<td style=\"width: 50%; height: 10px;\"><code>withBatchSize(number)<\/code><\/td>\n<td style=\"width: 50%; height: 10px;\">Batch size of a bulk insert.<\/td>\n<\/tr>\n<tr style=\"height: 49px;\">\n<td style=\"width: 50%; height: 49px;\"><code>setParallelism(number)<\/code><\/td>\n<td style=\"width: 50%; height: 49px;\">Number of concurrent connections to TiDB<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n<h4 class=\"wp-block-heading\"><strong>Note<\/strong>:&nbsp;&nbsp;<\/h4>\n\n\n\n<p>If you use the <code>ON DUPLICATE KEY UPDATE<\/code> clause in your SQL statement, please pay attention to the statement syntax. You should use the <code>VALUES(col_name)<\/code> function to refer to column values. For example:&nbsp;<\/p>\n\n\n<pre><code>INSERT INTO table (id, name) values (?, ?) ON DUPLICATE KEY UPDATE id=values(id)<\/code><\/pre>\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span><strong>Summary<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If you are writing a high-performance data ingestion program, we recommend that you read this article and test the parameters carefully. If you want to check whether BULK INSERT works, enable the TiDB general log to view the SQL statements.&nbsp;<\/p>\n\n\n\n<p>If you run into issues, feel free to join our<a href=\"https:\/\/slack.tidb.io\/invite?team=tidb-community&amp;channel=everyone&amp;ref=pingcap-blog\"> community on Slack<\/a> and TiDB Internals to share them with us. You can also <a href=\"https:\/\/www.pingcap.com\/contact-us\/\">request a demo<\/a> for these methods from PingCAP.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><strong>Keep reading: <\/strong><br><a href=\"https:\/\/www.pingcap.com\/blog\/analytics-on-tidb-cloud-with-databricks\/\">Analytics on TiDB Cloud with Databricks<\/a><br><a href=\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/\">Data Transformation on TiDB Made Easier<\/a><br><a href=\"https:\/\/www.pingcap.com\/blog\/analytics-on-tidb-cloud-with-databricks\/\">Using Airbyte to Migrate Data from TiDB Cloud to Snowflake<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB. <\/p>","protected":false},"author":176,"featured_media":5998,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ub_ctt_via":"","footnotes":""},"categories":[13],"tags":[39,111,29],"class_list":["post-5988","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-product","tag-best-practice","tag-tidb","tag-tutorial"],"acf":[],"featured_image_src":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg","author_info":{"display_name":"Fengbiao Liang","author_link":"https:\/\/www.pingcap.com\/ko\/blog\/author\/fengbiao-l\/"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Achieve High Performance Data Ingestion to TiDB in Apache Flink<\/title>\n<meta name=\"description\" content=\"This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Achieve High Performance Data Ingestion to TiDB in Apache Flink\" \/>\n<meta property=\"og:description\" content=\"This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:published_time\" content=\"2022-04-12T08:43:20+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-20T12:04:50+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-Flink-Social.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Fengbiao Liang\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-Flink-Social.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fengbiao Liang\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\"},\"author\":{\"name\":\"Fengbiao Liang\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/69b91b9095b79f8e16c7e24f28e3bc65\"},\"headline\":\"How to Achieve High-Performance Data Ingestion to TiDB in Apache Flink\",\"datePublished\":\"2022-04-12T08:43:20+00:00\",\"dateModified\":\"2024-12-20T12:04:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\"},\"wordCount\":503,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg\",\"keywords\":[\"Best Practice\",\"TiDB\",\"Tutorial\"],\"articleSection\":[\"Product\"],\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\",\"url\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\",\"name\":\"Achieve High Performance Data Ingestion to TiDB in Apache Flink\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg\",\"datePublished\":\"2022-04-12T08:43:20+00:00\",\"dateModified\":\"2024-12-20T12:04:50+00:00\",\"description\":\"This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg\",\"width\":2001,\"height\":668,\"caption\":\"TiDB-flink-banner\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to Achieve High-Performance Data Ingestion to TiDB in Apache Flink\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/69b91b9095b79f8e16c7e24f28e3bc65\",\"name\":\"Fengbiao Liang\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"caption\":\"Fengbiao Liang\"},\"url\":\"https:\/\/www.pingcap.com\/ko\/blog\/author\/fengbiao-l\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Achieve High Performance Data Ingestion to TiDB in Apache Flink","description":"This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pingcap.com\/ko\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/","og_locale":"ko_KR","og_type":"article","og_title":"Achieve High Performance Data Ingestion to TiDB in Apache Flink","og_description":"This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB.","og_url":"https:\/\/www.pingcap.com\/ko\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_published_time":"2022-04-12T08:43:20+00:00","article_modified_time":"2024-12-20T12:04:50+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-Flink-Social.jpg","type":"image\/jpeg"}],"author":"Fengbiao Liang","twitter_card":"summary_large_image","twitter_image":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-Flink-Social.jpg","twitter_creator":"@PingCAP","twitter_site":"@PingCAP","twitter_misc":{"Written by":"Fengbiao Liang","Est. reading time":"3\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#article","isPartOf":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/"},"author":{"name":"Fengbiao Liang","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/69b91b9095b79f8e16c7e24f28e3bc65"},"headline":"How to Achieve High-Performance Data Ingestion to TiDB in Apache Flink","datePublished":"2022-04-12T08:43:20+00:00","dateModified":"2024-12-20T12:04:50+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/"},"wordCount":503,"commentCount":0,"publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg","keywords":["Best Practice","TiDB","Tutorial"],"articleSection":["Product"],"inLanguage":"ko-KR","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/","url":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/","name":"Achieve High Performance Data Ingestion to TiDB in Apache Flink","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg","datePublished":"2022-04-12T08:43:20+00:00","dateModified":"2024-12-20T12:04:50+00:00","description":"This article shows you two ways you can achieve high performance when you need to quickly ingest streaming data from Apache Flink to TiDB.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/"]}]},{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#primaryimage","url":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg","width":2001,"height":668,"caption":"TiDB-flink-banner"},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"How to Achieve High-Performance Data Ingestion to TiDB in Apache Flink"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]},{"@type":"Person","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/69b91b9095b79f8e16c7e24f28e3bc65","name":"Fengbiao Liang","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/","url":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","caption":"Fengbiao Liang"},"url":"https:\/\/www.pingcap.com\/ko\/blog\/author\/fengbiao-l\/"}]}},"grav_blocks":false,"card_markup":"<a class=\"card-resource bg-white\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/how-to-achieve-high-performance-data-ingestion-to-tidb-in-apache-flink\/\"><div class=\"card-resource__image-container\"><img class=\"card-resource__image\" alt=\"TiDB-flink-banner\" src=\"https:\/\/static.pingcap.com\/files\/2022\/04\/TiDB-flink-banner-1.jpg\" loading=\"lazy\" width=2001 height=668 \/><\/div><div class=\"card-resource__content-container\"><div class=\"card-resource__content-head\"><div class=\"card-resource__category\">Product<\/div><\/div><h5 class=\"card-resource__title\">How to Achieve High-Performance Data Ingestion to TiDB in Apache Flink<\/h5><\/div><\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/5988","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/176"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/comments?post=5988"}],"version-history":[{"count":14,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/5988\/revisions"}],"predecessor-version":[{"id":24454,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/5988\/revisions\/24454"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media\/5998"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=5988"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/categories?post=5988"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/tags?post=5988"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}