{"id":17910,"date":"2024-06-26T08:28:06","date_gmt":"2024-06-26T15:28:06","guid":{"rendered":"https:\/\/www.pingcap.com\/?post_type=article&#038;p=17910"},"modified":"2024-06-26T08:28:08","modified_gmt":"2024-06-26T15:28:08","slug":"azure-openai-integration-with-tidb-vector-search","status":"publish","type":"article","link":"https:\/\/www.pingcap.com\/ko\/article\/azure-openai-integration-with-tidb-vector-search\/","title":{"rendered":"Azure OpenAI Integration with TiDB Vector Search: Enhancing Semantic Capabilities"},"content":{"rendered":"<p>In today&#8217;s data-driven world, businesses and developers seek advanced solutions to leverage vast amounts of data effectively. Integrating Azure OpenAI with TiDB&#8217;s <a href=\"\/ko\/ai\/\">Vector Search<\/a> capability presents an innovative approach to achieving semantic search and similarity search across various data types, such as text, images, and videos. This article explores the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Understanding_TiDB_Vector_Search\"><\/span>Understanding TiDB Vector Search<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>TiDB Vector Search enables semantic search, allowing users to find related data based on meaning rather than simple keyword matches. This is achieved through vector embeddings, which represent data as points in a semantic space. The distance between these points indicates their similarity, making it possible to perform tasks like image recognition, recommendation systems, and more.<\/p>\n\n\n\n<p><strong>Key Features of TiDB Vector Search:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Semantic Search:<\/strong> Search for data based on meaning, improving accuracy and relevance.<\/li>\n\n\n\n<li><strong>Versatility:<\/strong> Applicable to texts, images, videos, and other data types.<\/li>\n\n\n\n<li><strong>Unified Storage:<\/strong> Store data and their embeddings together in TiDB for seamless querying.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Azure_OpenAI_A_Brief_Overview\"><\/span>Azure OpenAI: A Brief Overview<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Azure OpenAI provides access to OpenAI&#8217;s powerful language models, enabling developers to integrate advanced natural language processing (NLP) capabilities into their applications. These models can generate embeddings that are crucial for semantic search applications.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Integrating_Azure_OpenAI_with_TiDB_Vector_Search\"><\/span>Integrating Azure OpenAI with TiDB Vector Search<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>By combining Azure OpenAI&#8217;s embedding generation with TiDB&#8217;s vector search, developers can create robust semantic search solutions. Here&#8217;s a step-by-step guide to setting up this integration.<\/p>\n\n\n\n<p><strong>1.Setting Up TiDB Serverless Cluster with Vector Search:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Sign Up:<\/strong> Create an account on TiDB Cloud.<\/li>\n\n\n\n<li><strong>Create Cluster:<\/strong> Follow the tutorial to create a TiDB Serverless cluster with vector support in the eu-central-1 region.<\/li>\n\n\n\n<li><strong>Connection Setup:<\/strong> Connect to the cluster using the provided connection details.<\/li>\n<\/ul>\n\n\n\n<p><strong>2.Generating Embeddings with Azure OpenAI:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Install Dependencies:<\/strong> Ensure you have Python 3.6+, and install necessary libraries.<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>pip install openai peewee pymysql tidb_vector<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Environment Configuration:<\/strong><\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>export OPENAI_API_KEY=\"your_openai_api_key\"\nexport TIDB_HOST=\"your_tidb_host\"\nexport TIDB_USERNAME=\"your_tidb_username\"\nexport TIDB_PASSWORD=\"your_tidb_password\"<\/code><\/pre>\n\n\n\n<p><strong>3.Inserting and Querying Data in TiDB:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Table Creation:<\/strong><\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>CREATE TABLE vector_table (id INT PRIMARY KEY, doc TEXT, embedding VECTOR(1536));<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Insert Data:<\/strong><\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>import openai\nfrom peewee import Model, MySQLDatabase, TextField\nfrom tidb_vector.peewee import VectorField\n\n# Initialize OpenAI client and TiDB connection\nclient = openai.OpenAI(api_key=\"your_openai_api_key\")\ndb = MySQLDatabase('test', user=\"your_tidb_username\", password=\"your_tidb_password\", host=\"your_tidb_host\", port=4000)\n\n# Define model\nclass DocModel(Model):\n    text = TextField()\n    embedding = VectorField(dimensions=1536)\n    class Meta:\n        database = db\n        table_name = \"vector_table\"\n\ndb.connect()\ndb.create_tables(&#91;DocModel])\n\n# Generate embeddings and insert data\ndocuments = &#91;\"Example text 1\", \"Example text 2\", \"Example text 3\"]\nembeddings = &#91;client.embeddings.create(input=doc, model=\"text-embedding-ada-002\").data for doc in documents]\ndata_source = &#91;{\"text\": doc, \"embedding\": emb&#91;'embedding']} for doc, emb in zip(documents, embeddings)]\nDocModel.insert_many(data_source).execute()<\/code><\/pre>\n\n\n\n<p><strong>4.Performing Semantic Search:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Query for Similar Data:<\/strong><\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>question = \"Find related examples\"\nquestion_embedding = client.embeddings.create(input=question, model=\"text-embedding-ada-002\").data&#91;0]&#91;'embedding']\nrelated_docs = DocModel.select(DocModel.text, DocModel.embedding.cosine_distance(question_embedding).alias(\"distance\")).order_by(SQL(\"distance\")).limit(3)\n\nfor doc in related_docs:\n    print(doc.distance, doc.text)<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Integrating Azure OpenAI with TiDB Vector Search unlocks powerful capabilities for semantic search and data retrieval. This combination enables developers to build advanced applications that understand and process data in a more meaningful way.<\/p>\n\n\n\n<p>Ready to explore the potential of TiDB and Azure OpenAI? Start by creating your TiDB Serverless cluster today at <a href=\"https:\/\/tidbcloud.com\/free-trial\/\">TiDB Cloud<\/a> and join the AI revolution in data management and semantic search.<\/p>","protected":false},"excerpt":{"rendered":"<p>In today&#8217;s data-driven world, businesses and developers seek advanced solutions to leverage vast amounts of data effectively. Integrating Azure OpenAI with TiDB&#8217;s Vector Search capability presents an innovative approach to achieving semantic search and similarity search across various data types, such as text, images, and videos. This article explores the synergy between Azure OpenAI and [&hellip;]<\/p>\n","protected":false},"author":8,"featured_media":0,"template":"","class_list":["post-17910","article","type-article","status-publish","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Azure OpenAI Integration with TiDB Vector Search<\/title>\n<meta name=\"description\" content=\"Explore the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.\" \/>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Azure OpenAI Integration with TiDB Vector Search\" \/>\n<meta property=\"og:description\" content=\"Explore the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/article\/azure-openai-integration-with-tidb-vector-search\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-26T15:28:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2024\/09\/11005522\/Homepage-Ad.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"714\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04\" \/>\n\t<meta name=\"twitter:data1\" content=\"3\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/\",\"url\":\"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/\",\"name\":\"Azure OpenAI Integration with TiDB Vector Search\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"datePublished\":\"2024-06-26T15:28:06+00:00\",\"dateModified\":\"2024-06-26T15:28:08+00:00\",\"description\":\"Explore the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Articles\",\"item\":\"https:\/\/www.pingcap.com\/article\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Azure OpenAI Integration with TiDB Vector Search: Enhancing Semantic Capabilities\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Azure OpenAI Integration with TiDB Vector Search","description":"Explore the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.","robots":{"index":"noindex","follow":"follow"},"og_locale":"ko_KR","og_type":"article","og_title":"Azure OpenAI Integration with TiDB Vector Search","og_description":"Explore the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.","og_url":"https:\/\/www.pingcap.com\/ko\/article\/azure-openai-integration-with-tidb-vector-search\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_modified_time":"2024-06-26T15:28:08+00:00","og_image":[{"width":1440,"height":714,"url":"https:\/\/static.pingcap.com\/files\/2024\/09\/11005522\/Homepage-Ad.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@PingCAP","twitter_misc":{"\uc608\uc0c1 \ub418\ub294 \ud310\ub3c5 \uc2dc\uac04":"3\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/","url":"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/","name":"Azure OpenAI Integration with TiDB Vector Search","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"datePublished":"2024-06-26T15:28:06+00:00","dateModified":"2024-06-26T15:28:08+00:00","description":"Explore the synergy between Azure OpenAI and TiDB Vector Search, providing a comprehensive guide to harnessing their combined power.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/article\/azure-openai-integration-with-tidb-vector-search\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"Articles","item":"https:\/\/www.pingcap.com\/article\/"},{"@type":"ListItem","position":3,"name":"Azure OpenAI Integration with TiDB Vector Search: Enhancing Semantic Capabilities"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]}]}},"card_markup":"        <a class=\"card-article\" href=\"https:\/\/www.pingcap.com\/ko\/article\/azure-openai-integration-with-tidb-vector-search\/\">            <h3>Azure OpenAI Integration with TiDB Vector Search: Enhancing Semantic Capabilities<\/h3>            <p>In today&#8217;s data-driven world, businesses and developers seek advanced solutions to leverage vast amounts of data effectively. Integrating Azure OpenAI with TiDB&#8217;s Vector Search capability presents an innovative approach to achieving semantic search and similarity search across various data types, such as text, images, and videos. This article explores the synergy between Azure OpenAI and [&hellip;]<\/p>        <\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/article\/17910","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/article"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/article"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/8"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=17910"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}