{"id":17274,"date":"2024-05-28T07:53:33","date_gmt":"2024-05-28T14:53:33","guid":{"rendered":"https:\/\/www.pingcap.com\/?post_type=article&#038;p=17274"},"modified":"2024-05-28T07:53:36","modified_gmt":"2024-05-28T14:53:36","slug":"building-rag-with-llama3-ollama-llamaindex-tidb-serverless","status":"publish","type":"article","link":"https:\/\/www.pingcap.com\/ko\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/","title":{"rendered":"Tutorial: Building a Retrieval-Augmented Generation System with Llama3, Ollama, LlamaIndex, and TiDB Serverless"},"content":{"rendered":"<p>Welcome to this step-by-step tutorial on creating a robust Retrieval-Augmented Generation (RAG) system using Llama3, Ollama, LlamaIndex, and <a href=\"https:\/\/tidb.cloud\/\">TiDB Serverless<\/a>, which is a MySQL-compatible database but with <a href=\"https:\/\/www.pingcap.com\/ko\/blog\/integrating-vector-search-into-tidb-for-ai-applications\/\">built-in vector storage<\/a> in it. This guide is designed to help you integrate these powerful technologies to leverage AI-driven search and response generation capabilities in your applications.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Prerequisites\"><\/span>Prerequisites<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Before you start, make sure you have:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Python 3.8 or later<\/li>\n\n\n\n<li>An account of TiDB Cloud with Vector Storage feature enabled to access TiDB Serverless according to this <a href=\"https:\/\/docs.google.com\/document\/d\/16zJADoPfmLNHP5tFqREOCslwmPUGhMIqXTL4QQ6vf2g\/edit#heading=h.ioilgixkp0q4\">guide<\/a>.<\/li>\n\n\n\n<li>Run local Llama3 with Ollama. You can find the setup guide <a href=\"https:\/\/replicate.com\/blog\/run-llama-locally\">\uc5ec\uae30<\/a>.<\/li>\n<\/ul>\n\n\n<div class=\"ub_call_to_action hide wp-block-ub-call-to-action-block\"  id=\"ub_call_to_action_90794c40-1441-4064-a49f-e25f75506427\">\n                <div class=\"ub_call_to_action_headline\">\n                    <p class=\"ub_call_to_action_headline_text\">Try TiDB Serverless with Vector Search<\/p><\/div>\n                <div class=\"ub_call_to_action_content\">\n                    <p class=\"ub_cta_content_text\">Join the waitlist for the private beta of built-in vector search.<\/p><\/div>\n                <div class=\"ub_call_to_action_button\">\n                    <a href=\"https:\/\/tidb.cloud\/ai\/\" target=\"_self\" rel=\"noopener noreferrer\"\n                        class=\"ub_cta_button\">\n                        <p class=\"ub_cta_button_text\">Join Now<\/p><\/a><\/div><\/div>\n\n\n<div class=\"ub_call_to_action\" id=\"ub_call_to_action_30278112-caff-4a2e-9096-b9e6f2fb5de6\">\n                <div class=\"ub_call_to_action_headline\">\n                    <p class=\"ub_call_to_action_headline_text\">Try TiDB Serverless with Vector Search<\/p><\/div>\n                <div class=\"ub_call_to_action_content\">\n                    <p class=\"ub_cta_content_text\">Join the waitlist for the private beta of built-in vector search.<\/p><\/div>\n                <div class=\"ub_call_to_action_button\">\n                    <a href=\"https:\/\/tidb.cloud\/ai\/\" target=\"_blank\" rel=\"noopener noreferrer\" class=\"ub_cta_button external-link\" data-gtag=\"event:go_to_lead_form_page,product_type:serverless,button_name:Join the Waitlist,position:article_middle_cta\">\n                        <p class=\"ub_cta_button_text\" data-gtag=\"event:go_to_lead_form_page,product_type:serverless,button_name:Join the Waitlist,position:article_middle_cta\">Join Now<\/p><\/a><\/div><\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_1_Setting_Up_Your_Environment\"><\/span>Step 1: Setting Up Your Environment<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>First, we need to install the necessary Python packages and configure our environment to connect to TiDB Serverless.<\/p>\n\n\n\n<p><strong>Install Packages<\/strong><\/p>\n\n\n\n<p>Open your terminal or command prompt and run the following commands to install the required packages:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip install llama-index-vector-stores-tidbvector\npip install llama-index<\/code><\/pre>\n\n\n\n<p><strong>Configure Database Connection<\/strong><\/p>\n\n\n\n<p>Set up your TiDB connection URL using environment variables for security:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import getpass\n\ntidb_connection_url = getpass.getpass(\"Enter your TiDB connection URL (format - mysql+pymysql:\/\/root@host:port\/database): \")<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_2_Using_Ollama_as_LLM\"><\/span>Step 2: Using Ollama as LLM<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Set up Llama3 with Ollama, a model from the Llama family, as your LLM for processing queries and generating responses. Make sure you have run <code>ollama run llama3<\/code> after starting the Ollama.<\/p>\n\n\n\n<p><strong>Configure LLM Settings<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from llama_index.llms.ollama import Ollama\nfrom llama_index.core import Settings\n\nSettings.llm = Ollama(model=\"llama3\", request_timeout=60.0)<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_3_Creating_a_Vector_Store_in_TiDB\"><\/span>Step 3: Creating a Vector Store in TiDB<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Now, let&#8217;s create a table in TiDB optimized for vector searching. This will store our data and make it searchable via semantic vectors.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from llama_index.vector_stores.tidbvector import TiDBVectorStore\n\nVECTOR_TABLE_NAME = \"your_vector_table_name\"\ntidbvec = TiDBVectorStore(\n    connection_string=tidb_connection_url,\n    table_name=VECTOR_TABLE_NAME,\n    distance_strategy=\"cosine\",\n    vector_dimension=1536,\n    drop_existing_table=False\n)<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Step_4_Implementing_RAG\"><\/span>Step 4: Implementing RAG<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Combine the retrieval capabilities of TiDB with the generation power of Ollama to answer queries.<\/p>\n\n\n\n<p><strong>Load Data and Create Index<\/strong><\/p>\n\n\n\n<p>Assuming you have some documents stored, load them and create an index:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>from llama_index.core import SimpleDirectoryReader, StorageContext, VectorStoreIndex\n\ndocuments = SimpleDirectoryReader(\".\/your_data_directory\").load_data()\nstorage_context = StorageContext.from_defaults(vector_store=tidbvec)\nindex = VectorStoreIndex.from_documents(documents, storage_context=storage_context)<\/code><\/pre>\n\n\n\n<p><strong>Query and Generate Response<\/strong><\/p>\n\n\n\n<p>Use the created index to handle a query and generate a response using the Ollama model:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>query_engine = index.as_query_engine()\nresponse = query_engine.query(\"What did the author discuss?\")\nprint(response)<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Congratulations! You&#8217;ve just set up a complete RAG system using Llama3, Ollama, LlamaIndex, and TiDB Serverless. This setup allows you to leverage advanced AI capabilities for semantic search and response generation, making your applications smarter and more responsive. Continue to experiment with different configurations and datasets to fully explore the potential of these tools.<\/p>\n\n\n\n<p><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-cyan-blue-color\"><em><strong>Start your journey with TiDB Serverless today and join the waitlist for TiDB Vector Search.<\/strong><\/em><\/mark><\/p>\n\n\n\n<p><a href=\"https:\/\/tidb.cloud\/ai\/\" class=\"button\" target=\"_blank\" data-gtag=\"event:go_to_lead_form_page,product_type:serverless,button_name:Join the Waitlist,position:article_bottom_cta\" rel=\"noopener\">Join the Waitlist<\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>Welcome to this step-by-step tutorial on creating a robust Retrieval-Augmented Generation (RAG) system using Llama3, Ollama, LlamaIndex, and TiDB Serverless, which is a MySQL-compatible database but with built-in vector storage in it. This guide is designed to help you integrate these powerful technologies to leverage AI-driven search and response generation capabilities in your applications. Prerequisites [&hellip;]<\/p>\n","protected":false},"author":8,"featured_media":0,"template":"","class_list":["post-17274","article","type-article","status-publish","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Building a RAG System with Llama3, Ollama, LlamaIndex and TiDB<\/title>\n<meta name=\"description\" content=\"Discover how to integrate powerful technologies such as Llama3 to leverage AI-driven search and response generation capabilities.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pingcap.com\/ko\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/\" \/>\n<meta property=\"og:locale\" content=\"ko_KR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building a RAG System with Llama3, Ollama, LlamaIndex and TiDB\" \/>\n<meta property=\"og:description\" content=\"Discover how to integrate powerful technologies such as Llama3 to leverage AI-driven search and response generation capabilities.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.pingcap.com\/ko\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/\" \/>\n<meta property=\"og:site_name\" content=\"TiDB\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/facebook.com\/pingcap2015\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-28T14:53:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/static.pingcap.com\/files\/2024\/09\/11005522\/Homepage-Ad.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"714\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@PingCAP\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2\ubd84\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/\",\"url\":\"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/\",\"name\":\"Building a RAG System with Llama3, Ollama, LlamaIndex and TiDB\",\"isPartOf\":{\"@id\":\"https:\/\/www.pingcap.com\/#website\"},\"datePublished\":\"2024-05-28T14:53:33+00:00\",\"dateModified\":\"2024-05-28T14:53:36+00:00\",\"description\":\"Discover how to integrate powerful technologies such as Llama3 to leverage AI-driven search and response generation capabilities.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/#breadcrumb\"},\"inLanguage\":\"ko-KR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.pingcap.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Articles\",\"item\":\"https:\/\/www.pingcap.com\/article\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Tutorial: Building a Retrieval-Augmented Generation System with Llama3, Ollama, LlamaIndex, and TiDB Serverless\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.pingcap.com\/#website\",\"url\":\"https:\/\/www.pingcap.com\/\",\"name\":\"TiDB\",\"description\":\"TiDB | SQL at Scale\",\"publisher\":{\"@id\":\"https:\/\/www.pingcap.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.pingcap.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"ko-KR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.pingcap.com\/#organization\",\"name\":\"PingCAP\",\"url\":\"https:\/\/www.pingcap.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ko-KR\",\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"contentUrl\":\"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png\",\"width\":811,\"height\":232,\"caption\":\"PingCAP\"},\"image\":{\"@id\":\"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/facebook.com\/pingcap2015\",\"https:\/\/x.com\/PingCAP\",\"https:\/\/linkedin.com\/company\/pingcap\",\"https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Building a RAG System with Llama3, Ollama, LlamaIndex and TiDB","description":"Discover how to integrate powerful technologies such as Llama3 to leverage AI-driven search and response generation capabilities.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pingcap.com\/ko\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/","og_locale":"ko_KR","og_type":"article","og_title":"Building a RAG System with Llama3, Ollama, LlamaIndex and TiDB","og_description":"Discover how to integrate powerful technologies such as Llama3 to leverage AI-driven search and response generation capabilities.","og_url":"https:\/\/www.pingcap.com\/ko\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/","og_site_name":"TiDB","article_publisher":"https:\/\/facebook.com\/pingcap2015","article_modified_time":"2024-05-28T14:53:36+00:00","og_image":[{"width":1440,"height":714,"url":"https:\/\/static.pingcap.com\/files\/2024\/09\/11005522\/Homepage-Ad.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_site":"@PingCAP","twitter_misc":{"Est. reading time":"2\ubd84"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/","url":"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/","name":"Building a RAG System with Llama3, Ollama, LlamaIndex and TiDB","isPartOf":{"@id":"https:\/\/www.pingcap.com\/#website"},"datePublished":"2024-05-28T14:53:33+00:00","dateModified":"2024-05-28T14:53:36+00:00","description":"Discover how to integrate powerful technologies such as Llama3 to leverage AI-driven search and response generation capabilities.","breadcrumb":{"@id":"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/#breadcrumb"},"inLanguage":"ko-KR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.pingcap.com\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pingcap.com\/"},{"@type":"ListItem","position":2,"name":"Articles","item":"https:\/\/www.pingcap.com\/article\/"},{"@type":"ListItem","position":3,"name":"Tutorial: Building a Retrieval-Augmented Generation System with Llama3, Ollama, LlamaIndex, and TiDB Serverless"}]},{"@type":"WebSite","@id":"https:\/\/www.pingcap.com\/#website","url":"https:\/\/www.pingcap.com\/","name":"\ud2f0DB","description":"TiDB | SQL at Scale","publisher":{"@id":"https:\/\/www.pingcap.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pingcap.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"ko-KR"},{"@type":"Organization","@id":"https:\/\/www.pingcap.com\/#organization","name":"PingCAP","url":"https:\/\/www.pingcap.com\/","logo":{"@type":"ImageObject","inLanguage":"ko-KR","@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/","url":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","contentUrl":"https:\/\/static.pingcap.com\/files\/2021\/11\/pingcap-logo.png","width":811,"height":232,"caption":"PingCAP"},"image":{"@id":"https:\/\/www.pingcap.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/facebook.com\/pingcap2015","https:\/\/x.com\/PingCAP","https:\/\/linkedin.com\/company\/pingcap","https:\/\/youtube.com\/channel\/UCuq4puT32DzHKT5rU1IZpIA"]}]}},"card_markup":"        <a class=\"card-article\" href=\"https:\/\/www.pingcap.com\/ko\/article\/building-rag-with-llama3-ollama-llamaindex-tidb-serverless\/\">            <h3>Tutorial: Building a Retrieval-Augmented Generation System with Llama3, Ollama, LlamaIndex, and TiDB Serverless<\/h3>            <p>Welcome to this step-by-step tutorial on creating a robust Retrieval-Augmented Generation (RAG) system using Llama3, Ollama, LlamaIndex, and TiDB Serverless, which is a MySQL-compatible database but with built-in vector storage in it. This guide is designed to help you integrate these powerful technologies to leverage AI-driven search and response generation capabilities in your applications. Prerequisites [&hellip;]<\/p>        <\/a>","_links":{"self":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/article\/17274","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/article"}],"about":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/types\/article"}],"author":[{"embeddable":true,"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/users\/8"}],"wp:attachment":[{"href":"https:\/\/www.pingcap.com\/ko\/wp-json\/wp\/v2\/media?parent=17274"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}