{"id":5012,"date":"2022-02-23T01:50:23","date_gmt":"2022-02-23T09:50:23","guid":{"rendered":"https:\/\/en.pingcap.com\/?p=5012"},"modified":"2024-06-06T21:45:28","modified_gmt":"2024-06-07T04:45:28","slug":"how-to-use-tidb-cloud-with-aws-glue-data-catalog","status":"publish","type":"post","link":"https:\/\/www.pingcap.com\/ko\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/","title":{"rendered":"How to Integrate TiDB Cloud with AWS Glue Data Catalog"},"content":{"rendered":"\n<p>Author: Andrew Ren (TiDB Cloud Solutions Architect at PingCAP)<\/p>\n\n\n\n<p>A data catalog is a collection of data metadata. The catalog is a glossary and inventory of available data across different data platforms such as databases, data warehouses, and data lakes. Data users, particularly analysts and data scientists, use it to help find specific data that they need. You can use the data catalog to store, annotate, and share metadata.<\/p>\n\n\n\n<p><a href=\"https:\/\/www.pingcap.com\/tidb-dedicated\/\">TiDB Cloud <\/a>is a fully managed cloud service of <a href=\"https:\/\/www.pingcap.com\/tidb\/\">TiDB<\/a>. The user experience is similar to Amazon Relational Database Service (RDS) and Google Cloud SQL. With some simple clicks in the UI, you can get a fully functional production-ready database in either Amazon Web Services (AWS) or Google Cloud Platform (GCP).<\/p>\n\n\n\n<p><a href=\"https:\/\/docs.aws.amazon.com\/glue\/latest\/dg\/populate-data-catalog.html\">AWS Glue<\/a><a href=\"https:\/\/docs.aws.amazon.com\/glue\/latest\/dg\/populate-data-catalog.html\"> Data<\/a><a href=\"https:\/\/docs.aws.amazon.com\/glue\/latest\/dg\/populate-data-catalog.html\"> Catalog<\/a> contains references to data that is used as sources and targets of your extract, transform, and load (ETL) jobs in AWS Glue. To create your data warehouse or data lake, you must catalog this data.<\/p>\n\n\n\n<p>This tutorial will walk you through how to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog. Major steps include:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Prepare your AWS account with the necessary networking and access configurations.<\/li>\n\n\n\n<li>Create a TiDB Cloud on demand cluster.<\/li>\n\n\n\n<li>Create an AWS Glue Data Catalog and link your TiDB Cloud cluster to it.<\/li>\n\n\n\n<li>Create test data and check out the data catalog.<\/li>\n\n\n\n<li>Clean up the test environment.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"579\" height=\"293\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72471.png\" alt=\"\" class=\"wp-image-5116\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72471.png 579w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72471-300x152.png 300w\" sizes=\"auto, (max-width: 579px) 100vw, 579px\" \/><\/figure>\n<\/div>\n\n\n<p class=\"has-text-align-center\"><em>TiDB Cloud and AWS Glue Data Catalog Architecture<\/em><\/p>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Before_you_begin\"><\/span>Before you begin<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Before you try the steps in this article, make sure you have:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A basic understanding of AWS<\/li>\n\n\n\n<li>An AWS account<\/li>\n\n\n\n<li>A TiDB Cloud account. (If you do not have a TiDB Cloud account, click <a href=\"https:\/\/tidbcloud.com\/free-trial\/\">here<\/a> to sign up for one.)<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Prepare_your_AWS_account\"><\/span><a><\/a>Prepare your AWS account<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The first step is to prepare your account for AWS Glue and AWS Glue Data Catalog.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><a><\/a>Create an S3 endpoint<\/h3>\n\n\n\n<p>Create an <a href=\"https:\/\/docs.aws.amazon.com\/vpc\/latest\/privatelink\/vpc-endpoints-s3.html\">Amazon Simple Storage Service (S3) endpoint<\/a>and attach it to the Amazon Virtual Private Cloud (VPC). This S3 endpoint is needed for Glue to export logs from your Glue workers in the VPC to S3.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to AWS VPC. In the left panel, click <strong>Endpoints<\/strong>.<\/li>\n\n\n\n<li>Click <strong>Create Endpoint<\/strong>.<\/li>\n\n\n\n<li>In the <strong>Service name<\/strong> field, search for <strong>s3;<\/strong> select the <strong>Gateway<\/strong> type, and create the endpoint.<\/li>\n<\/ol>\n\n\n\n<p>Please note your <strong>AWS account ID<\/strong>, <strong>VPC ID<\/strong>, and VPC\u2019s Classless Inter-Domain Routing <strong>(CIDR). You\u2019ll be entering them in a later step<\/strong>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><a><\/a>Create a security group<\/h3>\n\n\n\n<p>Create a security group in the VPC named <strong>glue_eni<\/strong>. Later, you will assign it to the Glue worker so that it has network access.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Create a security group, name it <strong>glue_eni.<\/strong><\/li>\n\n\n\n<li>Set security group inbound and outbound rules.\n<ol class=\"wp-block-list\" start=\"1\" style=\"list-style-type:lower-alpha\">\n<li>Specify a self-referencing inbound rule for all TCP ports, to allow AWS Glue components to communicate and also prevent access from other networks.<\/li>\n\n\n\n<li>Allow all outbound traffic. (These are the default settings.)<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\"><a><\/a><a><\/a>Prepare an IAM role<\/h3>\n\n\n\n<p>Prepare an Identity and Access Management (IAM) role to grant the necessary permissions to the Glue worker.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Create an IAM role, choose use case <strong>glue<\/strong>, and name it <strong>glue_iam<\/strong>.<\/li>\n\n\n\n<li>Assign the policy <strong>AWSGlueServiceRole<\/strong> to <strong>glue_iam.<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"241\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72472.png\" alt=\"\" class=\"wp-image-5118\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72472.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72472-300x116.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Create_a_TiDB_Cloud_on_demand_cluster\"><\/span><a><\/a><a><\/a>Create a TiDB Cloud on demand cluster<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Now that your AWS account is ready, create a TiDB Cloud cluster and connect it to your AWS VPC environment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><a><\/a>Set up a customized CIDR<\/h3>\n\n\n\n<p>Before you create a TiDB Cloud cluster, set up a customized CIDR for TiDB Cloud, so you can link the TiDB Cloud VPC and your own AWS VPC later via VPC Peering.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to your TiDB Cloud console and<\/li>\n\n\n\n<li>Click <strong>Network Access<\/strong>.<\/li>\n\n\n\n<li>Select <strong>Project CIDR<\/strong>.<\/li>\n\n\n\n<li>Input the CIDR. Make sure it\u2019s different from your existing AWS VPC\u2019s CIDR. Otherwise, you won\u2019t be able to VPC peer them.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Create a TiDB Cloud cluster<\/h3>\n\n\n\n<p>Now comes the exciting part: creating a TiDB Cloud cluster. Since it\u2019s just a test, you can create the smallest usable cluster size: one TiDB node and three TiKV nodes.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to your TiDB Cloud console. At the top right of the screen, click <strong>Create a cluster<\/strong>.<\/li>\n\n\n\n<li>Set <strong>Cluster Name<\/strong> to <strong>test.<\/strong><\/li>\n\n\n\n<li>Set your Root Password and note it for future use.<\/li>\n\n\n\n<li>Change the number of TiDB\/TiKV nodes and click <strong>Create<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"273\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72473.png\" alt=\"\" class=\"wp-image-5119\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72473.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72473-300x131.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Your TiDB Cloud cluster will be created in approximately 5 to 10 minutes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Connect TiDB Cloud to AWS VPC<a><\/a><\/h3>\n\n\n\n<p>Connect your TiDB Cloud environment to AWS VPC through VPC peering.<\/p>\n\n\n\n<ol type=\"1\">\n<li>Go to your TiDB Cloud console and click <strong>Network Access<\/strong>.<\/li>\n<li>Click <strong>VPC Peering<\/strong>.<\/li>\n<li>Click <strong>Add<\/strong> and configure VPC peering to <strong>my AWS VPC<\/strong>.<\/li>\n<li>Select <strong>AWS<\/strong> as the cloud provider.<\/li>\n<li>Input the <strong>AWS account ID<\/strong>, <strong>VPC ID<\/strong>, and its <strong>CIDR<\/strong>. (You can find this information in the AWS console.)<br><br><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-5147 size-full\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72474-1.png\" alt=\"\" width=\"628\" height=\"531\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72474-1.png 628w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72474-1-300x254.png 300w\" sizes=\"auto, (max-width: 628px) 100vw, 628px\" \/><br><br><\/li>\n<li>Click <strong>Initialize<\/strong>. You should see the following screen.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"307\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72475.png\" alt=\"\" class=\"wp-image-5121\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72475.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72475-300x148.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><a><\/a>Accept the invitation in AWS<\/h3>\n\n\n\n<p>VPC peering has been initialized, but there\u2019s no peering ID yet because you haven\u2019t accepted the invitation in your AWS account.<\/p>\n\n\n\n<ol type=\"1\">\n<li>In your AWS console, go to the VPC page.<\/li>\n<li>In the left panel, click <strong>Peering connections<\/strong>.<\/li>\n<li>Find the following VPC peering request.<br><br><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-5150 size-full\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72476-1.png\" alt=\"\" width=\"624\" height=\"111\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72476-1.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72476-1-300x53.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><br><br><\/li>\n<li>Click <strong>Accept request<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"233\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72477.png\" alt=\"\" class=\"wp-image-5123\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72477.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72477-300x112.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Verify the VPC peering status<\/h3>\n\n\n\n<p>Check that the VPC peering status is already active.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Return to your TiDB Cloud console.<\/li>\n\n\n\n<li>Click <strong>Network Access<\/strong>.<\/li>\n\n\n\n<li>Click <strong>VPC Peering<\/strong>. You should see the <strong>Status<\/strong> has changed into <strong>active<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"87\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72478.png\" alt=\"\" class=\"wp-image-5125\" style=\"width:624px;height:87px\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72478.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72478-300x42.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Now you have the two VPCs connected, but Glue workers can\u2019t yet access the TiDB Cloud cluster. To fully grant access, you need to add two more networking configurations on the AWS side.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Add networking configurations<\/h3>\n\n\n\n<p>In your AWS VPC\u2019s route tables, add routing to the VPC peering CIDR so that the VPC router knows where to send the traffic when its target is the TiDB Cloud\u2019s CIDR.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to the VPC page in your AWS console. In the left panel, click <strong>Route Tables<\/strong>.<\/li>\n\n\n\n<li>To enter the detail page, click on your route table\u2019s ID.<\/li>\n\n\n\n<li>On the right side of the screen, click <strong>Edit routes<\/strong>. Then click <strong>Add route<\/strong>.<\/li>\n\n\n\n<li>Set <strong>Destination<\/strong> as your TiDB Cloud CIDR.<\/li>\n\n\n\n<li>Set <strong>Target<\/strong> as the alias of your VPC peering. (It is prefixed with pcx-.)<\/li>\n\n\n\n<li>Click <strong>Save changes<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"133\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u72479.png\" alt=\"\" class=\"wp-image-5126\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72479.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u72479-300x64.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Allow inbound traffic<\/h3>\n\n\n\n<p>Next, in the glue_eni security group, allow all inbound traffic from the TiDB Cloud CIDR, so that the security group allows the traffic from Glue workers to the TiDB Cloud database.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to the VPC page in your AWS console. In the left panel, click <strong>Security Groups<\/strong>.<\/li>\n\n\n\n<li>To enter the detail page, find <strong>glue_eni<\/strong> and click on its ID.<\/li>\n\n\n\n<li>Click <strong>Edit inbound rules<\/strong>. Then click <strong>Add rule<\/strong>.<\/li>\n\n\n\n<li>Set <strong>Type<\/strong> as <strong>All traffic<\/strong>.<\/li>\n\n\n\n<li>Set <strong>Source<\/strong> as <strong>Custom<\/strong>.<\/li>\n\n\n\n<li>In the input box behind <strong>Source<\/strong>, enter your TiDB Cloud\u2019s CIDR.<\/li>\n\n\n\n<li>Click <strong>Save rules.<\/strong><\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"640\" height=\"81\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724710.png\" alt=\"\" class=\"wp-image-5127\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724710.png 640w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724710-300x38.png 300w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>The network setup is done. Once you know the endpoint of the TiDB cluster, you will be able to connect to it. <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Get the endpoint<\/h3>\n\n\n\n<p>Go to your TiDB console and click into the detail page of the TiDB cluster you just created.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Click <strong>Connect<\/strong>.<\/li>\n\n\n\n<li>Choose <strong>VPC Peering<\/strong> and click <strong>Creating Endpoint<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"317\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724711.png\" alt=\"\" class=\"wp-image-5128\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724711.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724711-300x152.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Make a note of the endpoint because you\u2019ll need this information to create the Data Catalog in the next section.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"85\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724712.png\" alt=\"\" class=\"wp-image-5129\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724712.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724712-300x41.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Create_the_Data_Catalog\"><\/span><a><\/a>Create the Data Catalog<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>AWS Glue can manage TiDB\u2019s metadata, but it needs to know where to look for the data. That\u2019s why you\u2019ll need to configure a database, a connection, and a crawler.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Add a database<\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to the AWS Glue console.<\/li>\n\n\n\n<li>In the left panel, click <strong>Database<\/strong>.<\/li>\n\n\n\n<li>Click <strong>Add database<\/strong>.<\/li>\n\n\n\n<li>Set <strong>Database name<\/strong> as <strong>tidb<\/strong>.<\/li>\n\n\n\n<li>Click <strong>Create<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"191\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724713.png\" alt=\"\" class=\"wp-image-5130\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724713.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724713-300x92.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Add a connection<\/h3>\n\n\n\n<ol type=\"1\">\n<li>In the left panel, click <strong>Connections<\/strong>.<\/li>\n<li>Click <strong>Add connection<\/strong>.<\/li>\n<li>Set <strong>Connection name<\/strong> as <strong>tidb<\/strong>.<\/li>\n<li>Set the <strong>Connection type<\/strong> as <strong>JDBC<\/strong>.<\/li>\n<li>Click <strong>Next<\/strong>.<\/li>\n<li>Configure the JDBC URL in the following format, replacing endpoint with the endpoint you noted in the previous section:&nbsp;<br><br>\n<pre><code>jdbc:mysql:\/\/[tidb cloud endpoint]:4000\/test<\/code><\/pre>\n<br><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-5154 size-full\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724714-1.png\" alt=\"\" width=\"624\" height=\"225\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724714-1.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724714-1-300x108.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><br><br><\/li>\n<li>Set <strong style=\"font-size: revert; color: initial;\">Username<\/strong><span style=\"font-size: revert; color: initial;\"> to <\/span><strong style=\"font-size: revert; color: initial;\">root<\/strong><span style=\"font-size: revert; color: initial;\">.<\/span><\/li>\n<li>Set <strong>Password<\/strong> to your TiDB Cloud cluster password.<\/li>\n<li>Choose the <strong>VPC<\/strong> and <strong>Subnet<\/strong>.<\/li>\n<li>Select <strong>glue_eni<\/strong> as the security group.<\/li>\n<li>Finish the creation flow.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"660\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724715.png\" alt=\"\" class=\"wp-image-5132\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724715.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724715-284x300.png 284w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Test the connection<\/h3>\n\n\n\n<p>To make sure your setup is correct, test the connection:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>In the connection list, select the connection you created.<\/li>\n\n\n\n<li>Click <strong>Test connection<\/strong>.<\/li>\n\n\n\n<li>You should see a green note in the console \u201ctidb connected successfully to your instance.\u201d<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Create a crawler<\/h3>\n\n\n\n<p>The Glue crawler crawls the metadata via the database connection. To create a crawler.<\/p>\n\n\n\n<ol type=\"1\">\n<li>In the left panel, click <strong>Crawlers<\/strong>.<\/li>\n<li>Click <strong>Add crawler<\/strong>.<\/li>\n<li>Set <strong>Crawler name<\/strong> as <strong>tidb.<\/strong><\/li>\n<li>Keep the default values for <strong>Crawler source type<\/strong> and <strong>Repeat crawls of S3 data stores<\/strong> and click <strong>Next<\/strong>.<\/li>\n<li>Set <strong>Choose a data store<\/strong> as <strong>JDBC<\/strong>.<\/li>\n<li>Set <strong>Connection<\/strong> as <strong>tidb.<\/strong><\/li>\n<li>Set <strong>Include path<\/strong> as <strong>test\/%<\/strong>.<\/li>\n<li>Click <strong>Next<\/strong>.<br><br><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-5160 size-full\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724716-1.png\" alt=\"\" width=\"624\" height=\"383\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724716-1.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724716-1-300x184.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><br><br><\/li>\n<li>Set <strong>IAM role<\/strong> as <strong>glue_iam<\/strong>.<\/li>\n<li>Click <strong>Next<\/strong>.<br><br><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-5161\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724717-1.png\" alt=\"\" width=\"624\" height=\"217\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724717-1.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724717-1-300x104.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><br><br><\/li>\n<li>Set <strong>Database<\/strong> as <strong>tidb<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"332\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724718.png\" alt=\"\" class=\"wp-image-5135\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724718.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724718-300x160.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Now all Glue setups are ready\u2014you have a database, a connection, and a crawler. Next, you\u2019ll need some test data to run the crawler and see what happens.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Create_a_test_data_in_the_TiDB_Cloud_cluster\"><\/span><a><\/a>Create a test data in the TiDB Cloud cluster<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Use TiDB Cloud\u2019s web shell feature to insert the test data. With this approach, you won\u2019t need to create extra EC2 instances.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to your TiDB Cloud console.<\/li>\n\n\n\n<li>Go to the TiDB cluster you created.<\/li>\n\n\n\n<li>Click <strong>Connect<\/strong>.<\/li>\n\n\n\n<li>Select <strong>Web SQL Shell<\/strong>.<\/li>\n\n\n\n<li>Click <strong>Open SQL Shell<\/strong>.<\/li>\n<\/ol>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"143\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724719.png\" alt=\"\" class=\"wp-image-5136\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724719.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724719-300x69.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>Run the following queries one by one to insert two tables into the database test.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Use test;<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>CREATE TABLE t1 (a int);<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>CREATE TABLE t2 (\n id BIGINT NOT NULL PRIMARY KEY auto_increment,\n b VARCHAR(200) NOT NULL\n);\n<\/code><\/pre>\n\n\n\n<p>Leave the web SQL shell open so that you can come back later to manipulate the schema and test how the data catalog picks up schema changes.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Check_the_Data_Catalog\"><\/span>Check the Data Catalog<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><a><\/a>Run the crawler<\/h3>\n\n\n\n<p>Run the crawler to collect metadata from the TiDB Cloud cluster.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Go to the AWS Glue console, and choose <strong>crawler<\/strong>.<\/li>\n\n\n\n<li>Select the <strong>tidb<\/strong> crawler.<\/li>\n\n\n\n<li>Click the <strong>Run crawler<\/strong> button.<\/li>\n<\/ol>\n\n\n\n<p>This example shows a crawler that was configured manually, so you can see how it works step by step. In production, you can always set it up to run on a schedule, so it can pick up your metadata changes automatically.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"69\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724720.png\" alt=\"\" class=\"wp-image-5137\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724720.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724720-300x33.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>After about two minutes, the crawler finishes its task.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"36\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724721.png\" alt=\"\" class=\"wp-image-5138\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724721.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724721-300x17.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Verify the results<\/h3>\n\n\n\n<p>Go to the tables and check the synchronization results. In the left panel, under <strong>Databases<\/strong>, click <strong>Tables<\/strong>. The two test tables you created are now in the Data Catalog.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"123\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724722.png\" alt=\"\" class=\"wp-image-5139\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724722.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724722-300x59.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>If you click into table <strong>t2<\/strong>, you\u2019ll be able to see it correctly recorded in the two columns that were created.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"228\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724723.png\" alt=\"\" class=\"wp-image-5140\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724723.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724723-300x110.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>You can add comments to the two columns in production to explain what these fields are for.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"103\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724724.png\" alt=\"\" class=\"wp-image-5141\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724724.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724724-300x50.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>To add a new column to table <strong>t2<\/strong>, go back to TiDB Cloud\u2019s web SQL shell and run the following query.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ALTER TABLE t2 ADD COLUMN c INT NOT NULL;<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<p>Now return to the AWS Glue console and run the TiDB crawler again.<\/p>\n\n\n\n<p>Afterwards, check the table details again, and you\u2019ll see that a new column is added. The comments on the old columns remain unchanged.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"224\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724725.png\" alt=\"\" class=\"wp-image-5142\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724725.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724725-300x108.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p>On the top right, click <strong>Compare versions<\/strong> to show the table versions and their differences.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"624\" height=\"265\" src=\"https:\/\/en.pingcap.com\/wp-content\/uploads\/2022\/02\/\u56fe\u724726.png\" alt=\"\" class=\"wp-image-5143\" srcset=\"https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724726.png 624w, https:\/\/static.pingcap.com\/files\/2022\/02\/\u56fe\u724726-300x127.png 300w\" sizes=\"auto, (max-width: 624px) 100vw, 624px\" \/><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Clean_up_the_test_environment\"><\/span><a><\/a>Clean up the test environment<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Make sure to clean up the test environment so that you don\u2019t get a surprise bill:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Delete the TiDB Cloud cluster.<\/li>\n\n\n\n<li>Delete the Glue crawler, connection, and database.<\/li>\n\n\n\n<li>Delete the <strong>glue_eni <\/strong>security group.<\/li>\n\n\n\n<li>Delete the <strong>glue_iam <\/strong>IAM role.<\/li>\n\n\n\n<li>Delete VPC peering and the route table rule.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span><a><\/a>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>TiDB Cloud and the AWS Glue Data catalog can work seamlessly together without any customization. This tutorial treated TiDB Cloud as a normal JDBC connection, and it worked out well. Using the networking configuration shown in this tutorial to connect TiDB Cloud and AWS Glue, you\u2019ll be able to automatically synchronize all metadata changes from your TiDB Cloud cluster to AWS Glue. Also, you can use the AWS Glue Data Catalog to annotate the metadata and manage metadata versions.<\/p>\n\n\n\n<p>Ready to <a href=\"http:\/\/tidbcloud.com\/free-trial\/\">give TiDB Cloud a try<\/a>? TiDB Cloud Developer Tier is now available. You can run a TiDB cluster for free for one year on Amazon Web Services. And make sure to follow us on <a href=\"https:\/\/twitter.com\/PingCAP\">Twitter<\/a> to stay updated on TiDB Cloud news!<\/p>\n\n\n\n<p>If you are interested in this topic and want to learn more, check out the following resources:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/docs.pingcap.com\/tidbcloud\/public-preview\">TiDB Cloud Documentation<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/docs.aws.amazon.com\/glue\/latest\/dg\/what-is-glue.html\">AWS Glue Documentation<\/a><\/li>\n<\/ul>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog. <\/p>","protected":false},"author":8,"featured_media":5064,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ub_ctt_via":"","footnotes":""},"categories":[13],"tags":[153,31,29],"class_list":["post-5012","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-product","tag-how-to","tag-tidb-cloud","tag-tutorial"],"acf":[],"featured_image_src":"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg","author_info":{"display_name":"TiDB Team","author_link":"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/"},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>How to Integrate TiDB Cloud with AWS Glue Data Catalog | TiDB<\/title>\n<meta name=\"description\" content=\"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Integrate TiDB Cloud with AWS Glue Data Catalog | TiDB\" \/>\n<meta property=\"og:description\" content=\"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:published_time\" content=\"2022-02-23T09:50:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-07T04:45:28+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2022\/02\/tidbCloud-AWSGlue-Social.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"900\" \/>\n\t<meta property=\"og:image:height\" content=\"471\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"TiDB Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:description\" content=\"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/static.pingcap.com\/files\/2022\/02\/tidbCloud-AWSGlue-Social.jpg\" \/>\n<meta name=\"twitter:creator\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"TiDB Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"14\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\"},\"author\":{\"name\":\"TiDB Team\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74\"},\"headline\":\"How to Integrate TiDB Cloud with AWS Glue Data Catalog\",\"datePublished\":\"2022-02-23T09:50:23+00:00\",\"dateModified\":\"2024-06-07T04:45:28+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\"},\"wordCount\":1995,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg\",\"keywords\":[\"How-to\",\"TiDB Cloud\",\"Tutorial\"],\"articleSection\":[\"Product\"],\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\",\"url\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\",\"name\":\"How to Integrate TiDB Cloud with AWS Glue Data Catalog | TiDB\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg\",\"datePublished\":\"2022-02-23T09:50:23+00:00\",\"dateModified\":\"2024-06-07T04:45:28+00:00\",\"description\":\"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg\",\"width\":1500,\"height\":500},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to Integrate TiDB Cloud with AWS Glue Data Catalog\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74\",\"name\":\"TiDB Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg\",\"caption\":\"TiDB Team\"},\"url\":\"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to Integrate TiDB Cloud with AWS Glue Data Catalog | TiDB","description":"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pingcap.com\/ko\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/","og_locale":"ko_KR","og_type":"article","og_title":"How to Integrate TiDB Cloud with AWS Glue Data Catalog | TiDB","og_description":"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.","og_url":"https:\/\/www.pingcap.com\/ko\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_published_time":"2022-02-23T09:50:23+00:00","article_modified_time":"2024-06-07T04:45:28+00:00","og_image":[{"width":900,"height":471,"url":"https:\/\/static.pingcap.com\/files\/2022\/02\/tidbCloud-AWSGlue-Social.jpg","type":"image\/jpeg"}],"author":"TiDB Team","twitter_card":"summary_large_image","twitter_description":"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.","twitter_image":"https:\/\/static.pingcap.com\/files\/2022\/02\/tidbCloud-AWSGlue-Social.jpg","twitter_creator":"@PingCAP","twitter_site":"@PingCAP","twitter_misc":{"Written by":"TiDB Team","Est. reading time":"14\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#article","isPartOf":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/"},"author":{"name":"TiDB Team","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74"},"headline":"How to Integrate TiDB Cloud with AWS Glue Data Catalog","datePublished":"2022-02-23T09:50:23+00:00","dateModified":"2024-06-07T04:45:28+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/"},"wordCount":1995,"commentCount":0,"publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg","keywords":["How-to","TiDB Cloud","Tutorial"],"articleSection":["Product"],"inLanguage":"ko-KR","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/","url":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/","name":"How to Integrate TiDB Cloud with AWS Glue Data Catalog | TiDB","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage"},"image":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage"},"thumbnailUrl":"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg","datePublished":"2022-02-23T09:50:23+00:00","dateModified":"2024-06-07T04:45:28+00:00","description":"This tutorial is to guide you to integrate TiDB Cloud with AWS Glue Data Catalog and manage TiDB metadata with the catalog.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/"]}]},{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#primaryimage","url":"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg","width":1500,"height":500},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"How to Integrate TiDB Cloud with AWS Glue Data Catalog"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]},{"@type":"Person","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/b17c1fde961eebd318de8729d595df74","name":"TiDB Team","image":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/person\/image\/","url":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","contentUrl":"https:\/\/static.pingcap.com\/files\/2022\/10\/17234942\/avatar.jpg","caption":"TiDB Team"},"url":"https:\/\/www.pingcap.com\/ko\/blog\/author\/pingcap\/"}]}},"grav_blocks":false,"card_markup":"<a class=\"card-resource bg-white\" href=\"https:\/\/www.pingcap.com\/ko\/blog\/how-to-use-tidb-cloud-with-aws-glue-data-catalog\/\"><div class=\"card-resource__image-container\"><img class=\"card-resource__image\" alt=\"How to Use TiDB Cloud with AWS Glue Catalog\" src=\"https:\/\/static.pingcap.com\/files\/2022\/02\/How-to-Use-TiDB-Cloud-with-AWS-Glue-Catalog.jpg\" loading=\"lazy\" width=1500 height=500 \/><\/div><div class=\"card-resource__content-container\"><div class=\"card-resource__content-head\"><div class=\"card-resource__category\">Product<\/div><\/div><h5 class=\"card-resource__title\">How to Integrate TiDB Cloud with AWS Glue Data Catalog<\/h5><\/div><\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/5012","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/comments?post=5012"}],"version-history":[{"count":65,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/5012\/revisions"}],"predecessor-version":[{"id":17559,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/posts\/5012\/revisions\/17559"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media\/5064"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=5012"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/categories?post=5012"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/tags?post=5012"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}