GraphRAG: How Knowledge Graphs Make RAG More Accurate

A support bot built on plain retrieval-augmented generation (RAG) can answer “What is our refund policy?” It struggles with “Which customers who churned last quarter also filed a refund request?” The first question matches a single chunk of text. The second requires connecting facts across multiple documents, something vector similarity search alone was never built to do. GraphRAG closes that gap by indexing entities and relationships as a knowledge graph alongside the usual vector embeddings, so an AI application can answer both kinds of questions from the same pipeline.

Overview of LLMs, RAG, and Knowledge Graphs

Brief explanation of Large Language Models (LLMs)

LLMs, such as GPT (Generative Pre-trained Transformer) series, are deep learning algorithms trained on vast amounts of text data. They can predict the next word in a sentence, making them powerful tools for generating human-like text based on the input they’re fed.

Short introduction to RAG

Retrieval-Augmented Generation (RAG) combines the generative power of LLMs with an external knowledge source, allowing the model to “retrieve” relevant information before “generating” a response. This process enhances the model’s ability to provide informed and precise answers.

Short Introduction to Knowledge Graphs

Knowledge Graphs organize and store complex data in an interconnected, graph-like structure. They enable logical reasoning over the stored data, facilitating precise information retrieval and data interconnectivity essential for various AI applications.

What Is GraphRAG?

GraphRAG is an approach, first introduced by Microsoft Research, that augments RAG by indexing a knowledge graph, entities and their relationships, alongside the usual vector embeddings. Instead of retrieving isolated chunks of text ranked by similarity, GraphRAG can traverse relationships between entities to answer questions that span multiple documents or facts.

The difference matters most on multi-hop questions. Asking “Which products did the customer who filed complaint #4021 previously return?” needs the retrieval system to connect a complaint to a customer to a return history, three separate facts a similarity search treats as unrelated text.

Why Plain RAG Falls Short on Connected Questions

Large language models generate fluent text but are limited by their training data. They produce outdated or irrelevant answers on anything requiring current or domain-specific knowledge. RAG addresses this by retrieving external context before generation, but the retrieval step still depends on vector similarity, which finds text that reads like the query, not text that is logically connected to it. Two mentions of the same customer in different documents can end up in entirely different regions of vector space if they’re phrased differently, so a plain RAG pipeline never connects them.

How TiDB Solves the Graph-Plus-Vector Storage Problem

Most GraphRAG implementations run two databases: a dedicated graph database for entities and relationships, and a separate vector database for embeddings, joined by application code. 티DB stores both in one MySQL-compatible schema. An entities table holds a native VECTOR column for embeddings; a relationships table holds foreign keys back to entities. Because both tables live in the same database, a single SQL query can traverse relationships and run a vector similarity search in one transaction, no separate graph store, no ETL pipeline syncing data between systems.

This single-schema approach is the same underlying vector and transactional infrastructure that powers production AI agent platforms on TiDB Cloud 스타터, including Kimi’s agent hosting platform, which provisions isolated databases for millions of concurrent agent-created applications. TiDB’s MySQL compatibility means the schema below works with standard SQL clients and ORMs, no proprietary graph query language to learn.

Building a GraphRAG Pipeline: Indexing, Retrieval, Reranking

A GraphRAG pipeline built this way runs in three stages.

Data indexing. An LLM extracts entities and relationships from source text, and both are written into TiDB alongside a vector embedding of each entity’s description. The schema:

CREATE TABLE `entities` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `name` varchar(512) DEFAULT NULL,
  `description` text DEFAULT NULL,
  `description_vec` vector<float> DEFAULT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;

CREATE TABLE `relationships` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `source_entity_id` int(11) DEFAULT NULL,
  `target_entity_id` int(11) DEFAULT NULL,
  `relationship_desc` text DEFAULT NULL,
  PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */,
  KEY `fk_1` (`source_entity_id`),
  KEY `fk_2` (`target_entity_id`),
  CONSTRAINT `fk_1` FOREIGN KEY (`source_entity_id`) REFERENCES `entities` (`id`),
  CONSTRAINT `fk_2` FOREIGN KEY (`target_entity_id`) REFERENCES `entities` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;

Because both tables are ordinary SQL tables, adding new entities or updating existing relationships as source data changes is a standard INSERT or UPDATE, not a separate graph-rebuild job. This matters for anything beyond a one-time demo: a support knowledge base or product catalog that changes daily needs the graph to stay current without a nightly batch re-index.

Data retrieval. Given a query, TiDB finds the nearest entity by vector distance, then joins through relationships to pull in connected entities:

WITH initial_entity AS (
    SELECT id FROM `entities`
    ORDER BY VEC_Cosine_Distance(description_vec, :embedding) LIMIT 1
), entities_ids AS (
    SELECT source_entity_id i FROM relationships r INNER JOIN initial_entity i ON r.target_entity_id = i.id
    UNION SELECT target_entity_id i FROM relationships r INNER JOIN initial_entity i ON r.source_entity_id = i.id
    UNION SELECT initial_entity.id i FROM initial_entity
) SELECT description FROM `entities` WHERE id IN (SELECT i FROM entities_ids);

Reranking. The retrieved entities are reranked for relevance to the query before being passed to the LLM for response generation.

A full working example, including the OpenAI embedding calls, is available in the GraphRAG demo on GitHub.

자주 묻는 질문

What is the difference between RAG and GraphRAG?

• RAG retrieves text chunks ranked by vector similarity alone.
• GraphRAG additionally indexes entities and relationships as a knowledge graph.
• This lets GraphRAG answer multi-hop questions that connect facts across documents, not just questions a single chunk can answer.

Do I need a separate graph database to build GraphRAG?

• No. A relational database with a native vector column, like TiDB, can store entities, relationships, and embeddings together.
• This avoids running and syncing two separate systems (a graph database and a vector database).
• Traversal queries run as standard SQL joins instead of a proprietary graph query language.

Is GraphRAG more expensive to run than vector-only RAG?

• Indexing costs more up front, since an LLM has to extract entities and relationships, not just generate embeddings.
• Query-time cost is comparable if entities and vectors live in one database, since retrieval is a single query instead of two round trips.
• The cost tradeoff is usually worth it for domains where multi-hop questions are common, like support history or compliance records.

Can I add GraphRAG to an existing RAG pipeline without rebuilding it?

• Yes, if the existing vector store can add a relationships table alongside the entities table already storing embeddings.
• Existing embeddings don’t need to be regenerated, only the entity-relationship extraction step is new.
• The GitHub demo linked above shows the schema addition needed on top of a standard entities table.

Get Started with GraphRAG on TiDB

GraphRAG works because it treats entities and relationships as first-class data, not just text to embed. Storing that graph and its vector embeddings in the same MySQL-compatible database removes the ETL layer most GraphRAG tutorials assume is necessary. TiDB Cloud 스타터, TiDB’s free tier, includes native vector search alongside standard SQL, so the schema in this piece runs without any additional infrastructure. Explore the full vector search and RAG pipelines documentation, or start with the GraphRAG demo on GitHub.