Key Takeaways
- Tune infrastructure before queries. Hardware, network, and cluster configuration set a ceiling no query rewrite can lift.
- Production TiKV nodes need 16+ cores, 64 GB+ of memory, and SSDs; TiFlash needs 48+ cores and 128 GB+.
- Use 10 GbE networking, ideally two cards per node. A 1 GbE link bottlenecks a growing cluster before CPU or memory does.
- Session variables, TiKV memory settings, and PD scheduling limits keep components from starving each other under load.
Tuning a slow query on top of undersized hardware or a congested network only buys back a fraction of the performance you are missing, because the ceiling sits at the infrastructure layer. This guide covers that layer specifically: hardware, network, and cluster-wide configuration for TiDB, before you get to individual query tuning.
What Should You Tune First in a TiDB Cluster?
Tune infrastructure before queries. Hardware sizing, network bandwidth, and cluster-wide TiDB, TiKV, and PD configuration set the ceiling that query-level optimization cannot exceed. Once you have sized and configured the cluster itself correctly, move to query-level performance tuning for further gains.
Understanding TiDB’s Architecture for Tuning
A TiDB cluster runs four main components, and each has a different hardware profile. That is why sizing happens per component rather than per cluster:
- TiDB Server: the stateless SQL layer, which parses queries, builds execution plans, and executes them. It is CPU- and memory-heavy and scales out by adding instances.
- TiKV: the distributed, transactional key-value storage engine. It stores data persistently and keeps it consistent through the Raft consensus algorithm, which makes disk and network performance decisive.
- Placement Driver (PD): manages cluster metadata, schedules data placement, and balances load across TiKV nodes. It needs less compute than the others but low-latency storage.
- TiFlash: the columnar storage engine that keeps a real-time replica of TiKV data for analytical queries. It carries the heaviest CPU and memory requirements in the cluster.
Hardware Requirements by Component
TiDB’s documented production minimums put the first number on every sizing decision. Treat them as floors, not targets: the docs recommend higher configurations for better performance.
| Component | CPU | Memory | Disk | Network | Min. Instances |
|---|---|---|---|---|---|
| TiDB | 16 cores+ | 48 GB+ | SSD | 10 GbE (2 preferred) | 2 |
| PD | 8 cores+ | 16 GB+ | SSD | 10 GbE (2 preferred) | 3 |
| TiKV | 16 cores+ | 64 GB+ | SSD (NVMe recommended) | 10 GbE (2 preferred) | 3 |
| TiFlash | 48 cores+ | 128 GB+ | 1 or more SSDs | 10 GbE (2 preferred) | 2 |
Production minimums from TiDB’s software and hardware requirements documentation (v8.5).
Storage decisions matter most on TiKV, where every write lands. Use NVMe SSDs, and keep each TiKV disk within 4 TB on PCIe SSDs or 1.5 TB on regular SSDs, since larger disks lengthen recovery and rebalancing. Keep TiKV disk usage below 80%. On AWS, Google Cloud, or Azure, put TiKV on cloud disks rather than instance store: instance store data disappears when the instance stops, restarts, or migrates.
Deploy TiFlash on different nodes from TiKV. If you must colocate them, add CPU cores and memory and put the two on separate disks, so replication and analytical scans do not compete for the same I/O. TiDB and PD can share a server in production, though separating them helps when performance and reliability requirements rise.
Network Configuration
Provision 10 GbE for production clusters, with two network cards per node where possible. TiDB’s own deployment guidance warns that gigabit cards run into bottlenecks, and they do so well before CPU or memory once a cluster grows past a handful of nodes. Bond the two cards for redundancy and added throughput.
Latency matters as much as raw bandwidth. Every committed write crosses nodes, because Raft needs a majority of replicas to confirm it, so network round-trip time lands directly on write latency. Distribute nodes across multiple availability zones for fault tolerance, and weigh the added cross-zone latency against the resilience it buys.
How Cluster Configuration Solves Resource Contention at Scale
As a cluster grows, TiDB, TiKV, and PD start competing for the same CPU, memory, and I/O, and the specific risk is one component starving another during peak load. Three groups of settings, one per component, carry most of that balance:
- TiDB: the tidb_distsql_scan_concurrency session variable caps how many concurrent scans a query can spawn, so one heavy query cannot take every available thread.
- TiKV: storage.block-cache.capacity and rocksdb.defaultcf.write-buffer-size divide memory between read caching and write buffering, cutting disk I/O without starving either path.
- PD: region-schedule-limit and leader-schedule-limit control how fast Regions and leaders move across nodes, so rebalancing after a node joins or fails does not itself become a source of contention.
Zhihu’s petabyte-scale migration to TiDB shows why these settings interact: at that scale, query concurrency and Region rebalancing compete for the same resources under sustained load, and you cannot tune either in isolation.
Set the Infrastructure Ceiling Before You Tune Queries
Hardware, network, and cluster configuration set the performance ceiling every query runs under. Getting them right first makes every later optimization worth more, and skipping them makes every later optimization worth less.
Start a free TiDB Cloud Starter cluster to skip infrastructure provisioning entirely for smaller workloads.
TiDB Cluster Performance Tuning FAQs
Should I Tune Infrastructure or Queries First in TiDB?
- Tune infrastructure first: hardware, network, and cluster configuration.
- Query-level tuning cannot exceed the ceiling infrastructure sets.
- Fix undersized memory or a congested network before touching SQL.
- Move to query optimization once you have sized the cluster itself correctly.
How Much Memory Does a TiDB Cluster Need?
- TiDB server: 48 GB+ per node in production.
- PD: 16 GB+ per node.
- TiKV: 64 GB+ per node.
- TiFlash: 128 GB+ per node.
- Requirements grow with data volume and query concurrency, so monitor memory use to catch paging early.
What Network Setup Does TiDB Recommend for Production?
- 10 GbE, with two network cards per node preferred.
- Bond the two cards for redundancy and added throughput.
- Spread nodes across multiple availability zones for fault tolerance.
- Treat latency as seriously as bandwidth: Raft puts a network round trip on every committed write.
Can I Run TiDB on HDDs?
- Not in production.
- Production requirements specify SSDs for TiDB, PD, TiKV, and TiFlash.
- The docs recommend NVMe SSDs for TiKV specifically.
- The docs list SAS storage only for the monitoring server.