{"id":6226,"date":"2022-04-22T04:58:03","date_gmt":"2022-04-22T11:58:03","guid":{"rendered":"https:\/\/en.pingcap.com\/?p=6226"},"modified":"2024-07-14T23:53:38","modified_gmt":"2024-07-15T06:53:38","slug":"data-transformation-on-tidb-made-easier","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/data-transformation-on-tidb-made-easier\/","title":{"rendered":"Data Transformation on TiDB Made Easier"},"content":{"rendered":"\n<p>Data build tool (<a href=\"https:\/\/www.getdbt.com\/\">dbt<\/a>) is a popular open-source data transformation tool that enables analytics engineers to transform data in their warehouses through SQL statements. The TiDB community recently released the <a href=\"https:\/\/github.com\/pingcap\/dbt-tidb\">dbt-tidb<\/a> adapter to make TiDB, a distributed SQL database to work with dbt. Through the dbt-tidb plug-in, analytics engineers working with TiDB can directly create forms and match data through SQL without having to think about the process of creating tables or views. They can also use Jinja, a dbt template language for writing SQL, test, package management, and other functions, which greatly improves efficiency.<\/p>\n<p>In this tutorial, I will show you how to use dbt with TiDB. Before you try any of the steps below, make sure the following items are installed:\u00a0<\/p>\n<ul>\n<li>TiDB 5.3 or later<\/li>\n<li>dbt 1.01 or later<\/li>\n<li>dbt-tidb 1.0.0<\/li>\n<\/ul>\n<h2><span class=\"ez-toc-section\" id=\"Installation\"><\/span>Installation<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>There are several ways you can install dbt and dbt-tidb, In this tutorial, we will use pypi. When you install dbt-tidb, dbt is installed as a dependency. So you only need one command to install both:<\/p>\n<pre><code>$ pip install dbt-tidb<\/code><\/pre>\n<p>You can also install dbt separately. Please refer to <a href=\"https:\/\/docs.getdbt.com\/dbt-cli\/install\/overview\">How to install dbt<\/a> in the dbt documentation.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Creating_the_project_jaffle_shop\"><\/span>Creating the project: jaffle shop<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>dbt-lab provides a project, jaffle_shop, to demonstrate dbt\u2019s functionality. You can get the project directly from GitHub:<\/p>\n<pre><code>$ git clone https:\/\/github.com\/dbt-labs\/jaffle_shop<\/code><br \/><code>$ cd jaffle_shop<\/code><\/pre>\n<p>All files in the jaffle_shop project directory are structured as follows.<\/p>\n<pre><code>ubuntu@ubuntu:~\/jaffle_shop$ tree<\/code><br \/><code>.<\/code><br \/><code>\u251c\u2500\u2500 dbt_project.yml<\/code><br \/><code>\u251c\u2500\u2500 etc<\/code><br \/><code>\u2502 \u251c\u2500\u2500 dbdiagram_definition.txt<\/code><br \/><code>\u2502 \u2514\u2500\u2500 jaffle_shop_erd.png<\/code><br \/><code>\u251c\u2500\u2500 LICENSE<\/code><br \/><code>\u251c\u2500\u2500 models<\/code><br \/><code>\u2502 \u251c\u2500\u2500 customers.sql<\/code><br \/><code>\u2502 \u251c\u2500\u2500 docs.md<\/code><br \/><code>\u2502 \u251c\u2500\u2500 orders.sql<\/code><br \/><code>\u2502 \u251c\u2500\u2500 overview.md<\/code><br \/><code>\u2502 \u251c\u2500\u2500 schema.yml<\/code><br \/><code>\u2502 \u2514\u2500\u2500 staging<\/code><br \/><code>\u2502 \u251c\u2500\u2500 schema.yml<\/code><br \/><code>\u2502 \u251c\u2500\u2500 stg_customers.sql<\/code><br \/><code>\u2502 \u251c\u2500\u2500 stg_orders.sql<\/code><br \/><code>\u2502 \u2514\u2500\u2500 stg_payments.sql<\/code><br \/><code>\u251c\u2500\u2500 README.md<\/code><br \/><code>\u2514\u2500\u2500 seeds<\/code><br \/><code>\u251c\u2500\u2500 raw_customers.csv<\/code><br \/><code>\u251c\u2500\u2500 raw_orders.csv<\/code><br \/><code>\u2514\u2500\u2500 raw_payments.csv<\/code><\/pre>\n<ul>\n<li><strong>dbt_project.yml<\/strong> is the dbt project configuration file, which holds the project name and database configuration file information.<\/li>\n<li><strong>The models directory<\/strong> contains the project\u2019s SQL models and table schemas. Note that the data analyst at your company writes this section. To learn more about models, see <a href=\"https:\/\/docs.getdbt.com\/docs\/building-a-dbt-project\/building-models\">dbt Docs<\/a>.<\/li>\n<li><strong>The seed directory<\/strong> stores CSV files that are dumped from database export tools. For example, TiDB can export the table data into CSV files through <a href=\"https:\/\/docs.pingcap.com\/tidb\/v4.0\/dumpling-overview\">Dumpling<\/a>. In the jaffle shop project, these CSV files are used as raw data to be processed.<\/li>\n<\/ul>\n<h2><span class=\"ez-toc-section\" id=\"Configuring_the_project\"><\/span>Configuring the project<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>To configure the project:<\/p>\n<ol>\n<li>Complete the global configuration. In the user directory, edit the default global profile, <code>~\/.dbt\/profiles.yml<\/code> to configure the connection with TiDB:\u00a0\n<pre><code>$ vi ~\/.dbt\/profiles.yml<\/code><br \/><code>jaffle_shop_tidb:           # project name<\/code><br \/><code>target: dev                # target<\/code><br \/><code>outputs:<\/code><br \/><code>   dev:<\/code><br \/><code>      type: tidb           # adapter type<\/code><br \/><code>      server: 127.0.0.1<\/code><br \/><code>      port: 4000<\/code><br \/><code>      schema: analytics    # database name<\/code><br \/><code>      username: root<\/code><br \/><code>      password: \"\"<\/code><\/pre>\n<\/li>\n<li>Complete the project configuration.<br \/>In the jaffle_shop project directory, enter the project configuration file <code>dbt_project.yml<\/code> and change the profile field to <code>jaffle_shop_tidb<\/code>. This configuration allows the project to query from the database as specified in the <code>~\/.dbt\/profiles.yml<\/code> file.\n<pre><br \/><code>$ cat dbt_project.yml<\/code><br \/><code>name: 'jaffle_shop'<\/code><br \/><br \/><code>config-version: 2<\/code><br \/><code>version: '0.1'<\/code><br \/><br \/><code>profile: 'jaffle_shop_tidb'      # note the modification here<\/code><br \/><br \/><code>model-paths: [\"models\"]          # model path<\/code><br \/><code>seed-paths: [\"seeds\"]            # seed path<\/code><br \/><code>test-paths: [\"tests\"] <\/code><br \/><code>analysis-paths: [\"analysis\"]<\/code><br \/><code>macro-paths: [\"macros\"]<\/code><br \/><br \/><code>target-path: \"target\"<\/code><br \/><code>clean-targets:<\/code><br \/><code>- \"target\"<\/code><br \/><code>- \"dbt_modules\"<\/code><br \/><code>- \"logs\"<\/code><br \/><br \/><code>require-dbt-version: [\"&gt;=1.0.0\", \"&lt;2.0.0\"]<\/code><br \/><br \/><code>models:<\/code><br \/><code>jaffle_shop:<\/code><br \/><code>materialized: table # *.sql which in models\/ would be materialized to table<\/code><br \/><code>staging: <\/code><br \/><code>materialized: view # *.sql which in models\/staging\/ would bt materialized to view<\/code><\/pre>\n<\/li>\n<li>Verify the configuration.<br \/>Run the following command to check whether the database and project configuration are correct:\n<pre><code>$ dbt debug\u200b\u200b<\/code><\/pre>\n<\/li>\n<\/ol>\n<h2><span class=\"ez-toc-section\" id=\"Loading_CSV_files\"><\/span>Loading CSV files<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Now that you have successfully created and configured the project, it\u2019s time to load the CSV data and materialize the CSV as a table in the target database. Note that this step is not generally required for a dbt project because the data items for processing are already in the database.<\/p>\n<ol>\n<li>Load the CSV files by running the following command:\n<pre><code style=\"white-space: pre-wrap;\">$ dbt seed<\/code><\/pre>\n<p>This displays the following:<\/p>\n<pre><code>Running with dbt=1.0.1<\/code><br \/><code>Partial parse save file not found. Starting full parse.<\/code><br \/><code>Found 5 models, 20 tests, 0 snapshots, 0 analyses, 172 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics<\/code><br \/><br \/><code>Concurrency: 1 threads (target='dev')<\/code><br \/><br \/><code>1 of 3 START seed file analytics.raw_customers.................................. [RUN]<\/code><br \/><code>1 of 3 OK loaded seed file analytics.raw_customers.............................. [INSERT 100 in 0.19s]<\/code><br \/><code>2 of 3 START seed file analytics.raw_orders..................................... [RUN]<\/code><br \/><code>2 of 3 OK loaded seed file analytics.raw_orders................................. [INSERT 99 in 0.14s]<\/code><br \/><code>3 of 3 START seed file analytics.raw_payments................................... [RUN]<\/code><br \/><code>3 of 3 OK loaded seed file analytics.raw_payments............................... [INSERT 113 in 0.24s]<\/code><\/pre>\n<p>As you can see in the results, the seed file was started and loaded into three tables: <code>analytics.raw_customers<\/code>, <code>analytics.raw_orders<\/code>, and <code>analytics.raw_payments<\/code>.<\/p>\n<\/li>\n<li>\n<p>Verify the results in TiDB. The show databases command lists the new analytics database that dbt created. The show tables command indicates that there are three tables in the analytics database, corresponding to the ones we created above.<\/p>\n<pre><code>mysql&gt; show databases;<\/code><br \/><code>+--------------------+<\/code><br \/><code>| Database |<\/code><br \/><code>+--------------------+<\/code><br \/><code>| INFORMATION_SCHEMA |<\/code><br \/><code>| METRICS_SCHEMA |<\/code><br \/><code>| PERFORMANCE_SCHEMA |<\/code><br \/><code>| analytics |<\/code><br \/><code>| mysql |<\/code><br \/><code>| test |<\/code><br \/><code>+--------------------+<\/code><br \/><code>6 rows in set (0.00 sec)<\/code><br \/><br \/><code>mysql&gt; show tables;<\/code><br \/><code>+---------------------+<\/code><br \/><code>| Tables_in_analytics |<\/code><br \/><code>+---------------------+<\/code><br \/><code>| raw_customers |<\/code><br \/><code>| raw_orders |<\/code><br \/><code>| raw_payments |<\/code><br \/><code>+---------------------+<\/code><br \/><code>3 rows in set (0.00 sec)<\/code><\/pre>\n<\/li>\n<\/ol>\n<h2><span class=\"ez-toc-section\" id=\"Running_the_dbt_project\"><\/span>Running the dbt project<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Now you are ready to run the configured projects and finish the data transformation.<\/p>\n<ol>\n<li>Run the dbt project to finish the data transformation:<br \/>\n<pre><code><span style=\"background-color: #f0f0f0; color: #1e1e1e; font-family: Menlo, Consolas, monaco, monospace; font-size: 14px;\">$ dbt run<\/span><\/code><\/pre>\n<pre><code>Running with dbt=1.0.1<\/code><br \/><code>Unable to do partial parsing because profile has changed<\/code><br \/><code>Unable to do partial parsing because a project dependency has been added<\/code><br \/><code>Found 5 models, 20 tests, 0 snapshots, 0 analyses, 172 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics<\/code><br \/><code>Concurrency: 1 threads (target='dev')<\/code><br \/><code>1 of 5 START view model analytics.stg_customers................................. [RUN]<\/code><br \/><code>1 of 5 OK created view model analytics.stg_customers............................ [SUCCESS 0 in 0.12s]<\/code><br \/><code>2 of 5 START view model analytics.stg_orders.................................... [RUN]<\/code><br \/><code>2 of 5 OK created view model analytics.stg_orders............................... [SUCCESS 0 in 0.08s]<\/code><br \/><code>3 of 5 START view model analytics.stg_payments.................................. [RUN]<\/code><br \/><code>3 of 5 OK created view model analytics.stg_payments............................. [SUCCESS 0 in 0.07s]<\/code><br \/><code>4 of 5 START table model analytics.customers.................................... [RUN]<\/code><br \/><code>4 of 5 OK created table model analytics.customers............................... [SUCCESS 0 in 0.16s]<\/code><br \/><code>5 of 5 START table model analytics.orders....................................... [RUN]<\/code><br \/><code>5 of 5 OK created table model analytics.orders.................................. [SUCCESS 0 in 0.12s]<\/code><\/pre>\nThe result shows three views (<code>analytics.stg_customers<\/code><span style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif; font-size: 18px;\">, <\/span><code>analytics.stg_orders<\/code><span style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif; font-size: 18px;\">, and <\/span><code>analytics.stg_payments<\/code><span style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif; font-size: 18px;\">) and two tables (<\/span><code>analytics.customers<\/code><span style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif; font-size: 18px;\"> and <\/span><code>analytics.orders<\/code><span style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif; font-size: 18px;\">) were created successfully.<\/span><\/li>\n<li>Go to the TiDB database to verify that the operation is successful.\n<pre><code>mysql&gt; show tables;<\/code><br \/><code>+---------------------+<\/code><br \/><code>| Tables_in_analytics |<\/code><br \/><code>+---------------------+<\/code><br \/><code>| customers     |<\/code><br \/><code>| orders        |<\/code><br \/><code>| raw_customers |<\/code><br \/><code>| raw_orders    |<\/code><br \/><code>| raw_payments  |<\/code><br \/><code>| stg_customers |<\/code><br \/><code>| stg_orders.   |<\/code><br \/><code>| stg_payments  |<\/code><br \/><code>+---------------------+<\/code><br \/><code>8 rows in set (0.00 sec)<\/code><br \/><br \/><code>mysql&gt; select * from customers;<\/code><br \/><code>+-------------+------------+-----------+-------------+-------------------+------------------+-------------------------+<\/code><br \/><code>| customer_id | first_name | last_name | first_order | most_recent_order | number_of_orders | customer_lifetime_value |<\/code><br \/><code>+-------------+------------+-----------+-------------+-------------------+------------------+-------------------------+<\/code><br \/><code>| 1 | Michael   | P. | 2018-01-01 | 2018-02-10 | 2 | 33.0000 |<\/code><br \/><code>| 2 | Shawn     | M. | 2018-01-11 | 2018-01-11 | 1 | 23.0000 |<\/code><br \/><code>| 3 | Kathleen  | P. | 2018-01-02 | 2018-03-11 | 3 | 65.0000 |<\/code><br \/><code>| 4 | Jimmy     | C. | NULL | NULL | NULL | NULL |<\/code><br \/><code>| 5 | Katherine | R. | NULL | NULL | NULL | NULL |<\/code><br \/><code>| 6 | Sarah     | R. | 2018-02-19 | 2018-02-19 | 1 | 8.0000 |<\/code><br \/><code>| 7 | Martin    | M. | 2018-01-14 | 2018-01-14 | 1 | 26.0000 |<\/code><br \/><code>| 8 | Frank     | R. | 2018-01-29 | 2018-03-12 | 2 | 45.0000 |<\/code><\/pre>\nThe output shows that five more tables or views have been added, and the data in the tables or views has been transformed. Note that only part of the data from the customer table is shown here.<\/li>\n<\/ol>\n<h2><span class=\"ez-toc-section\" id=\"Generating_visual_documents\"><\/span>Generating visual documents<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>dbt lets you generate visual documents that display the overall structure of the project and describe all the tables and views. To generate visual documents:<\/p>\n<ol>\n<li>Generate the document:\n<pre><code>$ dbt docs generate<br \/><\/code><\/pre>\n<\/li>\n<li>Start the server:\n<pre><code>$ dbt docs serve <\/code><br \/><code>Running with dbt=1.0.1<\/code><br \/><code>Serving docs at 0.0.0.0:8080<\/code><\/pre>\n<\/li>\n<li>\n<p><span style=\"color: initial; font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen-Sans, Ubuntu, Cantarell, 'Helvetica Neue', sans-serif;\">To access the document view from your browser, navigate to http:\/\/localhost:8080.<\/span><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-6230 size-full\" src=\"https:\/\/static.pingcap.com\/files\/2022\/04\/doc-view-dbt.png\" alt=\"doc view in DBT\" width=\"512\" height=\"263\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/04\/doc-view-dbt.png 512w, https:\/\/static.pingcap.com\/files\/2022\/04\/doc-view-dbt-300x154.png 300w\" sizes=\"auto, (max-width: 512px) 100vw, 512px\" \/><\/p>\n<\/li>\n<\/ol>\n<h2><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Currently, TiDB supports dbt in TiDB 4.0 and later versions. Earlier versions of TiDB may run into issues when working with dbt. For details, visit the <a href=\"https:\/\/github.com\/pingcap\/dbt-tidb\">tidb-dbt<\/a> project on GitHub. To get the most out of dbt, we recommend that you run TiDB 5.3 or later. These versions support all of dbt\u2019s functions.\u00a0<\/p>\n<p>If you run into issues, feel free to join our <a href=\"https:\/\/slack.tidb.io\/invite?team=tidb-community&amp;channel=everyone&amp;ref=pingcap-blog\">community on Slack<\/a> or file an <a href=\"https:\/\/github.com\/pingcap\/dbt-tidb\/issues\">issue<\/a> on our repository.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Thanks to a new adapter, TiDB can now work with the data build tool (dbt), a popular open source transformation tool. Now, analytics engineers working with TiDB can directly create forms and match data through SQL. This tutorial walks you through the new adapter and how to use dbt with TiDB. <\/p>","protected":false},"author":179,"featured_media":6231,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ub_ctt_via":"","footnotes":""},"categories":[18],"tags":[59,111],"class_list":["post-6226","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-community","tag-open-source","tag-tidb"],"acf":[],"featured_image_src":"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png","author_info":{"display_name":"Qiang Wu","author_link":"https:\/\/www.pingcap.com\/ko\/blog\/author\/wuqiang\/"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Data Transformation on TiDB Made Easier | TiDB<\/title>\n<meta name=\"description\" content=\"The TiDB community recently released the dbt-tidb adapter to make TiDB work with dbt(data build tool). Read and learn how to use it.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/data-transformation-on-tidb-made-easier\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Data Transformation on TiDB Made Easier | TiDB\" \/>\n<meta property=\"og:description\" content=\"The TiDB community recently released the dbt-tidb adapter to make TiDB work with dbt(data build tool). Read and learn how to use it.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/blog\/data-transformation-on-tidb-made-easier\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:published_time\" content=\"2022-04-22T11:58:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-07-15T06:53:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2022\/04\/20220422-195712.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1800\" \/>\n\t<meta property=\"og:image:height\" content=\"942\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Qiang Wu\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/static.pingcap.com\/files\/2022\/04\/20220422-195712.png\" \/>\n<meta name=\"twitter:creator\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Qiang Wu\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/\"},\"author\":{\"name\":\"Qiang Wu\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/6d5c9724aac811f92d08b1680044d3d4\"},\"headline\":\"Data Transformation on TiDB Made Easier\",\"datePublished\":\"2022-04-22T11:58:03+00:00\",\"dateModified\":\"2024-07-15T06:53:38+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/\"},\"wordCount\":729,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png\",\"keywords\":[\"Open source\",\"TiDB\"],\"articleSection\":[\"Community\"],\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/\",\"url\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/\",\"name\":\"Data Transformation on TiDB Made Easier | TiDB\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png\",\"datePublished\":\"2022-04-22T11:58:03+00:00\",\"dateModified\":\"2024-07-15T06:53:38+00:00\",\"description\":\"The TiDB community recently released the dbt-tidb adapter to make TiDB work with dbt(data build tool). Read and learn how to use it.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png\",\"width\":2824,\"height\":942},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Data Transformation on TiDB Made Easier\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/6d5c9724aac811f92d08b1680044d3d4\",\"name\":\"Qiang Wu\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"caption\":\"Qiang Wu\"},\"url\":\"https:\/\/www.pingcap.com\/ko\/blog\/author\/wuqiang\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Data Transformation on TiDB Made Easier | TiDB","description":"The TiDB community recently released the dbt-tidb adapter to make TiDB work with dbt(data build tool). Read and learn how to use it.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pingcap.com\/ko\/blog\/data-transformation-on-tidb-made-easier\/","og_locale":"ko_KR","og_type":"article","og_title":"Data Transformation on TiDB Made Easier | TiDB","og_description":"The TiDB community recently released the dbt-tidb adapter to make TiDB work with dbt(data build tool). Read and learn how to use it.","og_url":"https:\/\/www.pingcap.com\/ko\/blog\/data-transformation-on-tidb-made-easier\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_published_time":"2022-04-22T11:58:03+00:00","article_modified_time":"2024-07-15T06:53:38+00:00","og_image":[{"width":1800,"height":942,"url":"https:\/\/static.pingcap.com\/files\/2022\/04\/20220422-195712.png","type":"image\/png"}],"author":"Qiang Wu","twitter_card":"summary_large_image","twitter_image":"https:\/\/static.pingcap.com\/files\/2022\/04\/20220422-195712.png","twitter_creator":"@PingCAP","twitter_site":"@PingCAP","twitter_misc":{"Written by":"Qiang Wu","Est. reading time":"7\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#article","isPartOf":{"@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/"},"author":{"name":"Qiang Wu","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/6d5c9724aac811f92d08b1680044d3d4"},"headline":"Data Transformation on TiDB Made Easier","datePublished":"2022-04-22T11:58:03+00:00","dateModified":"2024-07-15T06:53:38+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/"},"wordCount":729,"commentCount":0,"publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png","keywords":["Open source","TiDB"],"articleSection":["Community"],"inLanguage":"ko-KR","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/","url":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/","name":"Data Transformation on TiDB Made Easier | TiDB","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png","datePublished":"2022-04-22T11:58:03+00:00","dateModified":"2024-07-15T06:53:38+00:00","description":"The TiDB community recently released the dbt-tidb adapter to make TiDB work with dbt(data build tool). Read and learn how to use it.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/"]}]},{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#primaryimage","url":"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png","width":2824,"height":942},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/blog\/data-transformation-on-tidb-made-easier\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"Data Transformation on TiDB Made Easier"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]},{"@type":"Person","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/6d5c9724aac811f92d08b1680044d3d4","name":"Qiang Wu","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/","url":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","caption":"Qiang Wu"},"url":"https:\/\/www.pingcap.com\/ko\/blog\/author\/wuqiang\/"}]}},"grav_blocks":false,"card_markup":"<a class=\"card-resource bg-white\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/data-transformation-on-tidb-made-easier\/\"><div class=\"card-resource__image-container\"><img class=\"card-resource__image\" alt=\"tidb-dbt\" src=\"https:\/\/static.pingcap.com\/files\/2022\/04\/tidb-dbt.png\" loading=\"lazy\" width=2824 height=942 \/><\/div><div class=\"card-resource__content-container\"><div class=\"card-resource__content-head\"><div class=\"card-resource__category\">Community<\/div><\/div><h5 class=\"card-resource__title\">Data Transformation on TiDB Made Easier<\/h5><\/div><\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/6226","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/179"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/comments?post=6226"}],"version-history":[{"count":17,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/6226\/revisions"}],"predecessor-version":[{"id":18129,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/6226\/revisions\/18129"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media\/6231"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=6226"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/categories?post=6226"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/tags?post=6226"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}