Vector Search Meets Distributed SQL: Why Agentic AI Does Not Need Another Database

Key Takeaways

• Every system added to an AI stack holds a copy or a slice of the same data, and every copy needs its own ingestion path, its own operators, and its own sync job.
• A vector is meaning encoded as numbers. Comparing a billion of them exhaustively is not affordable, which is why approximate nearest neighbor indexes exist.
• TiDB feeds its columnar store, vector index, and full-text index through the same Raft stream that keeps the row store consistent.
• The SQL optimizer decides which store answers which part of a query. Developers do not tag storage engines, and they do not write the query plan.
• Copy-on-write branching gives an agent a writable copy of production data without duplicating it.

Add a vector database to the existing stack. Sync it. Maintain it. Debug it when it drifts. Teams building agentic applications have largely accepted that sequence as the price of admission for working with embeddings, and the fourth step is the one worth pausing on. Drift is not an incidental bug waiting to be fixed. It is the predictable result of storing the same data in two systems that update independently.

At TiDB SCaiLE Europe 2026, Mattias Jonsson, Principal Software Engineer at TiDB, made the case that the separate vector store is not the price of building with AI. It is a workaround for a constraint that has since moved. The argument has two halves: the multi-system stack costs considerably more than it appears to, in sync pipelines, operational expertise, and developer confusion, and vector search, columnar analytics, and full-text search are all capabilities a distributed SQL database can now hold natively, kept consistent by the replication protocol already running underneath it.

Why the AI Stack Multiplies Into Data Sync Hell

The problem is not that any one of these systems is bad. It is that they overlap. The same data, or fragments of it, lands in all of them, which creates three costs that compound with every addition.

The first is synchronization. Data reaches each system through a different ingestion path, usually a brittle extract-transform-load pipeline that adds, removes, or decomposes data along the way. Those pipelines have to stay correct, because a query that touches several systems has to return one answer. Nobody wants a result where one system reflects the update just written and another does not.

The second is operational burden. Each system needs its own expertise to run well. Four systems mean either engineers who know all four deeply or four teams who each know one, plus additional people responsible for the pipelines between them. Change one system and its pipelines change, which propagates work to every system downstream.

The third is developer decision paralysis. Which system holds the answer to this question? Where does this update go? Who keeps it consistent? The result is a Frankenstack: a set of components assembled into something that runs but that nobody fully reasons about. Total cost of ownership rises across licensing, infrastructure, and operations, and it multiplies rather than adds with each new system.

Mattias Jonsson, Principal Software Engineer at TiDB, presenting at TiDB SCaiLE Europe 2026

What a Vector Is and Why Exhaustive Comparison Does Not Scale

A vector is meaning encoded as numbers. An embedding model takes an input, a phrase or an image or a document, and returns a fixed-length list of floating point values. Because every output from a given model has the same shape, comparison becomes arithmetic, which is the thing computers do well. The largest OpenAI embedding models produce roughly 3,000 dimensions per object.

The reason this matters is that text comparison alone fails at meaning. “Which automobile has the highest velocity” and “which car is fastest” ask the same question and share almost no characters. A literal string comparison finds nothing. Vectors close that gap, which is why similarity search sits at the foundation of AI applications rather than at the edge.

Vector search also runs in plain SQL. A normal SELECT that orders results by distance, commonly cosine distance, compares the embedding of a question against the embeddings stored in every candidate row and returns the closest matches.

The catch is scale. Roughly 3,000 numbers per comparison is trivial for a handful of rows. A million rows is a small database by production standards, and a large one starts at a billion. A billion comparisons across 3,000 dimensions each is expensive in both compute and time, and exact indexing does not rescue it, because high-dimensional vector data cannot be indexed exactly the way an integer column can. What works instead is approximation. Approximate nearest neighbor algorithms, HNSW among them, accept that a result set might miss an element or return the sixth-closest match instead of the fourth, in exchange for making the search efficient across a billion rows.

Why Vector Search Belongs in the Database That Already Holds the Data

Once approximate indexing makes vector search practical, the architectural question becomes obvious. The production data is already in the SQL database. If the vectors live there too, most of the stack’s cost disappears.

The write becomes a single write to a single store, with no pipeline and nothing to keep in sync. The query becomes a single query in a single language over a single connection, rather than graph query syntax for one system, a bespoke API for the vector store, and hand-built glue to combine partial results from each. The data is always fresh, because there is only one copy. Full-text search can live in the same place. Every pipeline removed is also a failure surface removed, which counts for more in production than it does on an architecture diagram.

Most importantly, the developer stops being the query planner. In a multi-system stack, somebody has to decide which system to hit first, how to combine what comes back, whether the steps can run in parallel, and what to do when data has not landed in one system yet. That is query optimization, performed manually, without transactional guarantees. SQL was designed to remove exactly that work: a query asks for a result rather than specifying how to retrieve it, and the optimizer produces the execution plan, combines and ranks the data, and does it over a transactionally consistent view.

That optimizer is also the hardest component in a database to build, which is the joke the session used to make the point land:

“If you flunk the database optimizer course, that is when we send you to rocket science.” 
– Mattias Jonsson, Principal Software Engineer, TiDB

Rebuilding a worse version of it by hand, across four systems, is a regression rather than an architecture.

How TiDB Keeps Row, Columnar, Vector, and Full-Text Stores Consistent

TiDB’s answer is that all of these access patterns are served by one system, kept consistent by one mechanism. The row store is the transactional store where ordinary indexes live, replicated three ways by default through the Raft protocol.

The insight that unlocked the rest was that Raft already carries a stream of every change. That stream can feed additional stores, each shaped for a different kind of question:

• A columnar store for analytics, which can scan a single column across a billion rows quickly and handles aggregations and scans where no index applies.
• A vector index, which is a subsystem rather than a per-row structure, maintaining its own approximate index for similarity search.
• A full-text index, built on the same pattern.

Because all of them are fed by the same replication stream, all of them present a consistent view of the same data. And because the SQL layer knows which store answers which part of a query, a single query can draw on several of them without the application tagging engines or hinting at storage. The developer writes SQL. The optimizer routes it.

How TiDB X Moves the Architecture Onto Object Storage

TiDB X, the architecture behind TiDB Cloud, rearranges these components into services built on object storage as the source of truth, with effectively unlimited bandwidth. Query processing sits on top, with row and column engines caching active data locally and leaving cold data in the object store.

The structural gain is that maintenance work moves out of the query path. The row store uses a log-structured merge tree, which needs compaction to stay current and reclaim space. Previously that work ran inside each of the three replicas while they were also serving queries. With object storage it runs once, on nodes that exist only while the job does, and the result is cached back. The same applies to schema changes and statistics collection: ALTER TABLE reads from and writes to the object store directly, then hands the finished index to new queries, and the services doing that work only exist during the operation.

Customers moving from the previous generation to TiDB X have cut costs by roughly half while gaining performance and lowering latency, and the architecture can scale compute to zero when nothing is running.

Why Copy-on-Write Branching Matters for Agents

Branching is the feature that turns this architecture into something agents can use directly. A branch is a point-in-time copy of a database created through copy-on-write, so it costs almost nothing and completes in minutes rather than duplicating the data. An agent, or an engineer, gets a full writable copy of production data, can experiment against it, and can throw it away.

What that removes is the compromise everyone has accepted for years: testing against an extracted sample, or stale data, or data massaged into a usable shape. Production stays safe while the experiment runs against the real thing. For a business, that compresses time to market and shortens the iteration loop. For agents, it means an isolated environment per task, at a cost that makes disposable environments reasonable.

One System Doing More, Not More Systems

The direction of travel is not a database plus an AI stack. It is a single system doing more. A SQL database that speaks vectors, scales horizontally, scans columns for analytics, searches text, holds agent memory, and branches on demand covers what agentic AI actually needs from a data layer, and it does it with one source of truth that is fresh and consistent by construction rather than by pipeline.

The multi-system stack was a reasonable response to databases that could not do these things. That constraint has moved. For teams designing an AI platform now, the question worth asking before adding the fourth system is whether the first one can already answer it.

See how TiDB supports agentic AI workloads 그리고 start a TiDB Cloud cluster to run vector search against your own data in a single query.