In the rapidly evolving landscape of machine learning and database technologies, combining the strengths of different tools can lead to innovative solutions. One such powerful combination is using Jina AI’s embedding capabilities with TiDB’s vector search functionality. This blog will guide you through building a semantic cache service using Jina AI Embeddings and TiDB Vector.<\/p>\n\n\n\n
A semantic cache stores the results of expensive queries and reuses them when the same or similar queries are made. This type of cache uses semantic understanding rather than exact key matching, making it particularly useful in applications requiring natural language processing or similar complex data retrieval tasks.<\/p>\n\n\n\n
Prerequisites<\/strong><\/p>\n\n\n\n Ensure you have the following installed:<\/p>\n\n\n\n First, set up your environment configuration. Create a Install the necessary Python packages:<\/p>\n\n\n\n Use Create the engine and the database schema.<\/p>\n\n\n\n Set up the FastAPI application and endpoints for setting and getting cache entries.<\/p>\n\n\n\n Implement a function to get embeddings from Jina AI<\/a>.<\/p>\n\n\n\n Prerequisites<\/strong><\/p>\n\n\n\n Run the example<\/strong><\/p>\n\n\n\n 1.Clone this repo<\/strong><\/p>\n\n\n\n 2.Create a virtual environment<\/strong><\/p>\n\n\n\n 3.Install dependencies<\/strong><\/p>\n\n\n\n 4.Set the environment variables<\/strong><\/p>\n\n\n\n Get the or create a 5.Run this example<\/strong><\/p>\n\n\n\n Start the semantic cache server<\/strong><\/p>\n\n\n\n 6.Test the API<\/strong><\/p>\n\n\n\n Get the Jina AI API key from the Jina AI Embedding API<\/a> page, and save it somewhere safe for later use.<\/p>\n\n\n\n By combining Jina AI’s powerful embedding capabilities with TiDB’s efficient vector operations, you can build a robust semantic cache service. This service is ideal for applications requiring fast, intelligent caching and retrieval of semantically similar data. Start experimenting with this setup to explore its full potential in your projects.<\/p>\n\n\n\n There are some examples to show how to use the tidb-vector-python to interact with TiDB Vector in different scenarios.<\/p>\n\n\n\n Happy coding!<\/p>\n","protected":false},"excerpt":{"rendered":" In the rapidly evolving landscape of machine learning and database technologies, combining the strengths of different tools can lead to innovative solutions. One such powerful combination is using Jina AI’s embedding capabilities with TiDB’s vector search functionality. This blog will guide you through building a semantic cache service using Jina AI Embeddings and TiDB Vector. […]<\/p>\n","protected":false},"author":8,"featured_media":0,"template":"","class_list":["post-17767","article","type-article","status-publish","hentry"],"acf":[],"yoast_head":"\n\n
<\/span>Step-by-Step Implementation<\/span><\/h2>\n\n\n\n
1.Configuration<\/h4>\n\n\n\n
.env<\/code> file to store your database URI and TTL (Time to Live) settings.<\/p>\n\n\n\nDATABASE_URI=mysql+pymysql:\/\/<username>:<password>@<host>:<port>\/<database>?ssl_mode=VERIFY_IDENTITY&ssl_ca=\/etc\/ssl\/cert.pem\nTIME_TO_LIVE=604800 # Default is 1 week<\/code><\/pre>\n\n\n\n2.Install Required Libraries<\/h4>\n\n\n\n
pip install fastapi requests sqlmodel sqlalchemy python-dotenv tidb-vector<\/code><\/pre>\n\n\n\n3.Define the Cache Model<\/h4>\n\n\n\n
SQLModel<\/code> to define your cache model, incorporating vector fields and automatic timestamping.<\/p>\n\n\n\nfrom sqlmodel import SQLModel, Field, Column, DateTime, String, Text\nfrom sqlalchemy import func\nfrom tidb_vector.sqlalchemy import VectorType\nfrom typing import Optional\nfrom datetime import datetime\n\nclass Cache(SQLModel, table=True):\n __table_args__ = {\n # Setting the TTL (Time to Live) for the cache entries\n 'mysql_TTL': f'created_at + INTERVAL {TIME_TO_LIVE} SECOND',\n }\n\n id: Optional[int] = Field(default=None, primary_key=True)\n key: str = Field(sa_column=Column(String(255), unique=True, nullable=False))\n key_vec: Optional[list[float]] = Field(\n sa_column=Column(\n VectorType(768), # Define the vector type with 768 dimensions\n default=None,\n comment=\"hnsw(distance=l2)\", # Using HNSW (Hierarchical Navigable Small World) algorithm for distance calculation\n nullable=False,\n )\n )\n value: Optional[str] = Field(sa_column=Column(Text))\n created_at: datetime = Field(\n sa_column=Column(DateTime, server_default=func.now(), nullable=False)\n )\n updated_at: datetime = Field(\n sa_column=Column(DateTime, server_default=func.now(), onupdate=func.now(), nullable=False)\n )<\/code><\/pre>\n\n\n\n4.Create the Database Engine<\/h4>\n\n\n\n
from sqlmodel import create_engine\n\n# Create the engine using the database URI\nengine = create_engine(DATABASE_URI)\n# Create all tables in the database\nSQLModel.metadata.create_all(engine)<\/code><\/pre>\n\n\n\n5.FastAPI Setup<\/h4>\n\n\n\n
from fastapi import FastAPI, Depends\nfrom fastapi.security import HTTPBearer, HTTPAuthorizationCredentials\nfrom sqlmodel import Session, select\n\n# Initialize FastAPI app\napp = FastAPI()\nsecurity = HTTPBearer()\n\n@app.post(\"\/set\")\ndef set_cache(\n credentials: HTTPAuthorizationCredentials = Depends(security),\n cache: Cache\n):\n # Generate embeddings for the given key using Jina AI\n cache.key_vec = generate_embeddings(credentials.credentials, cache.key)\n with Session(engine) as session:\n session.add(cache)\n session.commit()\n return {'message': 'Cache has been set'}\n\n@app.get(\"\/get\/{key}\")\ndef get_cache(\n credentials: HTTPAuthorizationCredentials = Depends(security),\n key: str,\n max_distance: Optional[float] = 0.1,\n):\n # Generate embeddings for the given key using Jina AI\n key_vec = generate_embeddings(credentials.credentials, key)\n # The max value of distance is 0.3\n max_distance = min(max_distance, 0.3)\n\n with Session(engine) as session:\n result = session.exec(\n select(\n Cache,\n Cache.key_vec.cosine_distance(key_vec).label('distance')\n ).order_by(\n 'distance'\n ).limit(1)\n ).first()\n\n if result is None:\n return {\"message\": \"Cache not found\"}, 404\n\n cache, distance = result\n if distance > max_distance:\n return {\"message\": \"Cache not found\"}, 404\n\n return {\n \"key\": cache.key,\n \"value\": cache.value,\n \"distance\": distance\n }<\/code><\/pre>\n\n\n\n6.Generate Embeddings<\/h4>\n\n\n\n
import requests\nimport os\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\ndef generate_embeddings(jinaai_api_key: str, text: str):\n JINAAI_API_URL = 'https:\/\/api.jina.ai\/v1\/embeddings'\n JINAAI_HEADERS = {\n 'Content-Type': 'application\/json',\n 'Authorization': f'Bearer {jinaai_api_key}'\n }\n JINAAI_REQUEST_DATA = {\n 'input': [text],\n 'model': 'jina-embeddings-v2-base-en' # Use the Jina Embeddings model with 768 dimensions\n }\n response = requests.post(JINAAI_API_URL, headers=JINAAI_HEADERS, json=JINAAI_REQUEST_DATA)\n # Extract and return the embedding from the response\n return response.json()['data'][0]['embedding']<\/code><\/pre>\n\n\n\n<\/span>How to Use This App<\/span><\/h2>\n\n\n\n
\n
\n
git clone https:\/\/github.com\/pingcap\/tidb-vector-python.git<\/code><\/pre>\n\n\n\n\n
cd tidb-vector-python\/examples\/semantic-cache\npython3 -m venv .venv\nsource .venv\/bin\/activate<\/code><\/pre>\n\n\n\n\n
pip install -r requirements.txt<\/code><\/pre>\n\n\n\n\n
HOST<\/code>, PORT<\/code>, USERNAME<\/code>, PASSWORD<\/code>, and DATABASE<\/code> from the TiDB Cloud console, as described in the [Prerequisites](..\/README.md#prerequisites) section. Then set the following environment variables:<\/p>\n\n\n\nexport DATABASE_URI=\"mysql+pymysql:\/\/<USERNAME>:<PASSWORD>@<HOST>:<PORT>\/<DATABASE>?ssl_ca=\/etc\/ssl\/cert.pem&ssl_verify_cert=true&ssl_verify_identity=true\"<\/code><\/pre>\n\n\n\n.env<\/code> file with the above environment variables.<\/p>\n\n\n\n\n
uvicorn cache:app --reload<\/code><\/pre>\n\n\n\n\n
\n
POST \/set<\/code><\/li>\n<\/ul>\n\n\n\ncurl --location ':8000\/set' \\\n--header 'Content-Type: application\/json' \\\n--header 'Authorization: Bearer <your jina token>' \\\n--data '{\n \"key\": \"what is tidb\",\n \"value\": \"tidb is a mysql-compatible and htap database\"\n}'<\/code><\/pre>\n\n\n\n\n
GET \/get\/<key><\/code><\/li>\n<\/ul>\n\n\n\ncurl --location ':8000\/get\/what%27s%20tidb%20and%20tikv?max_distance=0.5' \\\n--header 'Content-Type: application\/json' \\\n--header 'Authorization: Bearer <your jina token>'<\/code><\/pre>\n\n\n\n<\/span>Conclusion<\/span><\/h2>\n\n\n\n
More Demos<\/h3>\n\n\n\n
\n