How Vitess Works Internally: VTGate Query Routing, VTTablet Management, and the Sharding Middleware That Scales MySQL to Planetary Proportions

MySQL is one of the most battle-tested databases in existence. It is also, by design, a single-node system. When your write throughput saturates one server, you cannot simply add another MySQL instance and expect queries to spread. That is the problem Vitess was built to solve.

Vitess sits in front of MySQL and provides horizontal scaling through sharding, connection pooling, and query routing without requiring application-level awareness of where data lives. It powers YouTube’s database layer and underpins PlanetScale’s managed offering. Understanding how it works internally tells you when and why to reach for it.

The Three-Layer Architecture

Vitess decomposes the scaling problem into three distinct components that never overlap in responsibility.

VTGate is the stateless query router. Your application connects to VTGate over the MySQL wire protocol, sends a standard SQL query, and VTGate decides which shard or shards need to answer it. There is no state in VTGate itself. You can run dozens of VTGate instances behind a load balancer and lose any of them without consequence.

VTTablet is a sidecar process that runs alongside each MySQL instance. Every MySQL server in a Vitess cluster has exactly one VTTablet managing it. VTTablet handles connection pooling from VTGate to that MySQL instance, enforces query timeouts and table ACLs, performs health checks, manages schema changes, and participates in replication topology. Applications never talk to MySQL directly; they always go through VTGate which proxies through VTTablet.

The topology service stores the cluster’s metadata: which keyspaces exist, how shards map to tablet instances, which tablet is currently the primary for each shard. Vitess supports etcd or ZooKeeper as the topology backend. This is the source of truth that VTGate reads on startup and watches for changes. Topology data is read-heavy and small in volume; a few hundred bytes per tablet entry.

The separation is deliberate. VTGate scales horizontally because it is stateless. VTTablet runs on the same host as MySQL and owns that server’s lifecycle. The topology service is a small, durable coordination layer that does not sit in the query path.

VSchema: Mapping Logic to Physics

Before VTGate can route a query, it needs to understand how logical tables map to physical shards. This mapping lives in the VSchema, a JSON configuration stored in the topology service.

A keyspace in VSchema terms is a logical database that may span one shard (unsharded) or many shards (sharded). For a sharded keyspace you define:

• A sharding key (called the keyspace ID) for each table, typically a column like user_id or order_id.
• A vindex (vindexes are Vitess’s term for a sharding function): consistent hashing, unicode case-insensitive, or a custom lookup table.

// Simplified TypeScript representation of a VSchema definition
interface VSchema {
  keyspaces: {
    [keyspace: string]: {
      sharded: boolean;
      vindexes: {
        [vindexName: string]: {
          type: "hash" | "unicode_loose_md5" | "lookup_unique";
          params?: Record<string, string>;
          owner?: string;
        };
      };
      tables: {
        [tableName: string]: {
          column_vindexes: Array<{
            column: string;
            name: string; // refers to vindex name above
          }>;
          auto_increment?: {
            column: string;
            sequence: string;
          };
        };
      };
    };
  };
}

// Example: orders table sharded by customer_id using consistent hash
const schema: VSchema = {
  keyspaces: {
    commerce: {
      sharded: true,
      vindexes: {
        hash: { type: "hash" },
        customer_lookup: {
          type: "lookup_unique",
          params: {
            table: "customer_index",
            from: "customer_email",
            to: "customer_id",
          },
          owner: "customers",
        },
      },
      tables: {
        orders: {
          column_vindexes: [
            { column: "customer_id", name: "hash" },
          ],
        },
        customers: {
          column_vindexes: [
            { column: "customer_id", name: "hash" },
            { column: "email", name: "customer_lookup" },
          ],
        },
      },
    },
  },
};

The hash vindex maps a keyspace ID to a keyrange. Each shard owns a contiguous range of the keyspace, for example 00-40, 40-80, 80-c0, c0-00. When VTGate receives a query with WHERE customer_id = 12345, it hashes the value, identifies the owning keyrange, and routes to the single shard that holds that data.

Lookup vindexes handle non-primary sharding columns. A lookup vindex is itself a table that maps an alternative key (email) to the keyspace ID. VTGate performs a lookup query to resolve the keyspace ID before routing the main query. These lookups go to their own shard and add a round trip, which is the explicit tradeoff for supporting non-primary key lookups without scatter.

Query Rewriting and Scatter-Gather

When a query cannot be pinned to a single shard, VTGate performs a scatter-gather operation. It fans the query out to all relevant shards in parallel, collects results, merges them, and returns a single response to the application.

Consider a query without a sharding key predicate:

SELECT order_id, total, created_at
FROM orders
WHERE status = 'pending'
ORDER BY created_at DESC
LIMIT 20;

VTGate has no keyspace ID to route on. It rewrites the query for each shard, removes the LIMIT 20 clause (since each shard needs to return its top 20 candidates), collects all shard results, performs an in-memory merge-sort, and applies the limit on the aggregated set. The rewritten per-shard query becomes:

SELECT order_id, total, created_at
FROM orders
WHERE status = 'pending'
ORDER BY created_at DESC
LIMIT 20; -- vtgate may increase this or send without limit depending on memory config

For aggregation queries like COUNT(*) or SUM(amount), VTGate rewrites to request the aggregate from each shard and combines numerically. GROUP BY on a sharding key stays local to one shard. GROUP BY on a non-sharding column becomes a cross-shard scatter followed by a merge.

Cross-shard JOIN is the hardest case. Vitess supports two strategies: scatter the join to all shards and merge (expensive), or use a lookup vindex to route both sides of the join to the same shard (only works when join columns map to the same keyspace). Queries that genuinely need data from multiple shards on different keyspaces require the application to handle the join in code, or use a materialized view built with VReplication.

VReplication: The Unified CDC Engine

VReplication is Vitess’s change-data-capture layer. It is not a bolt-on feature; it is the engine that powers live resharding, cross-cluster replication, and materialized views.

A VReplication stream reads the MySQL binary log from a source tablet and applies changes to a target tablet using standard MySQL DML. The stream tracks its position in the binlog and handles restarts idempotently. Each stream has a VGTID (Vitess Global Transaction ID) that gives a consistent, cross-shard position.

The flow for any VReplication-based operation is:

1. Copy phase: VReplication does a consistent snapshot of the source table using SELECT ... LOCK IN SHARE MODE in batches, inserting rows into the target. This runs at a configurable copy batch size to limit impact on the source.
2. Catch-up phase: While copying, the binlog keeps streaming. After the copy completes, VReplication replays any binlog events that accumulated during the copy.
3. Running phase: Once caught up to within a configurable lag threshold (default 10 seconds), the stream enters the running state and applies changes in near-real time.

This three-phase approach means you can run VReplication on a live production table without downtime.

MoveTables and Reshard: Live Migration Without Downtime

Resharding in Vitess is a first-class workflow, not a manual procedure. Two operations cover the common cases.

MoveTables migrates one or more tables from one keyspace to another. This handles the common migration from an unsharded keyspace to a sharded one. VReplication copies the data, keeps it current, then a Traffic switch atomically moves read and write traffic to the new location.

Reshard splits existing shards. If you have two shards (-80 and 80-) and need four (-40, 40-80, 80-c0, c0-), Reshard sets up VReplication streams from each source shard to the appropriate target shards (each source feeds two targets), runs the copy and catch-up phases, then performs a traffic cutover.

The traffic switch in both workflows is controlled at the VTGate level through the topology service. Reads can be switched independently from writes (switch reads first, validate, then switch writes). The source shards remain available for rollback until you explicitly delete them. This gives you a validation window where you can run queries against both old and new shards and compare results.

// Workflow states a Reshard operation moves through
type ReshardState =
  | "NotStarted"
  | "Copying"        // VReplication copy phase running
  | "Running"        // Caught up, streaming changes
  | "SwitchingReads" // VTGate routing reads to new shards
  | "SwitchingWrites"// VTGate routing writes to new shards
  | "Done"           // Old shards can be dropped
  | "RolledBack";    // Reverted to old shards

// VTGate evaluates shard routing at query time from topology cache
interface ShardRoutingRule {
  from_keyspace: string;
  to_keyspace: string;
  shard: string;
}

Online DDL: Schema Changes at Scale

Schema changes on large MySQL tables are painful. ALTER TABLE on a 500 GB table takes hours and locks the table or causes replication lag. Vitess Online DDL wraps gh-ost or pt-online-schema-change (you choose which tool per migration) to perform shadow table migrations, then cuts over with a brief lock window.

When you submit an Online DDL migration through Vitess:

1. VTGate routes the DDL to the primary tablet of each shard.
2. Each VTTablet runs the migration asynchronously using the chosen tool.
3. Progress and status are tracked in internal Vitess tables (_vt.schema_migrations).
4. You can pause, resume, or cancel in-flight migrations.

Because each shard runs independently, a four-shard cluster performs four simultaneous gh-ost runs. This is faster than running serially but means your schema change tooling needs to handle parallel execution correctly.

Vitess also has a built-in vitess strategy that uses VReplication itself as the migration engine, removing the dependency on gh-ost entirely. This is the recommended approach in recent Vitess versions.

Connection Pooling and Tablet Health

VTTablet maintains two connection pools per MySQL instance: a transaction pool for connections that are inside an explicit transaction, and a normal pool for stateless queries. The transaction pool is typically smaller because idle transactions hold locks and should be short-lived.

Upstream, VTGate keeps a small number of connections to each VTTablet (not one per application connection). The multiplexing happens at VTGate: thousands of application MySQL connections map to a few dozen VTTablet connections per shard. This is where Vitess achieves the connection consolidation that makes running thousands of application pods viable against MySQL.

VTGate continuously monitors tablet health through a tablet health check. Each VTTablet broadcasts a health stream that reports replication lag, whether the tablet considers itself healthy, and whether it is a primary or replica. VTGate uses this stream to:

• Route read queries to healthy replicas only (based on configurable lag tolerance).
• Detect primary failovers and update routing within seconds.
• Remove tablets that stop streaming from the serving set.

When a primary fails and a new one is elected (Vitess uses vtorc, its own orchestration layer built on the older Orchestrator project), VTGate picks up the topology change from the topology service and redirects writes to the new primary. The combination of health streaming and topology watches gives VTGate sub-10-second failover detection in typical deployments.

Production Considerations

Choosing a sharding key is irreversible in practice. You can reshard to change the number of shards, but changing the column you shard on requires a full MoveTables migration with no shortcut. Choose a key with high cardinality and uniform distribution. Sequential IDs with a hash vindex work well. Timestamp-based sharding creates hot shards.

Scatter queries will surprise you. Any query without a sharding key predicate fans out to every shard. On a 16-shard cluster, a simple SELECT COUNT(*) FROM orders issues 16 parallel queries. Monitor vtgate_queries_processed by plan type; a high ratio of Scatter plans against large shard counts signals missing vindex coverage.

VTTablet sizing follows MySQL sizing. VTTablet is not memory-heavy on its own. Size your instances for MySQL’s buffer pool, not for VTTablet overhead. A typical VTTablet process uses 200-400 MB of RAM on top of MySQL.

Topology service availability is critical. VTGate caches topology but refreshes it continuously. If etcd or ZooKeeper becomes unavailable, VTGate continues serving traffic from cache, but tablet health updates stop. A prolonged topology outage means VTGate cannot detect primary failovers. Run your topology service with the same availability SLO as your database cluster.

Test your traffic switch in staging. The Reshard and MoveTables cutover steps are atomic but not instantaneous. There is a brief window where in-flight transactions are held. A traffic switch on a high-write keyspace can cause a transaction queue buildup. Always measure max write latency during a traffic switch in staging before executing in production.

Tradeoffs Comparison

	Vitess	CockroachDB	TiDB	Citus	PlanetScale	Plain MySQL Replication
Storage engine	MySQL	Custom (Pebble/RocksDB)	TiKV (RocksDB)	PostgreSQL	MySQL (via Vitess)	MySQL
SQL compatibility	MySQL dialect	PostgreSQL dialect	MySQL dialect	PostgreSQL dialect	MySQL dialect	MySQL
Horizontal writes	Sharding middleware	Native distributed	Native distributed	Sharding extension	Sharding middleware	No
Cross-shard transactions	2PC (limited, avoid)	Serializable ACID	Optimistic/Pessimistic ACID	No (manual)	2PC (limited)	No
Schema migrations	Online DDL via VReplication/gh-ost	Built-in online	Built-in online	Manual or pg tools	Online DDL	Manual/pt-osc
Operational complexity	High (many components)	Medium	Medium-High	Medium	Low (managed)	Low
Resharding	Live via VReplication	Automatic rebalancing	Automatic region rebalancing	Manual shard splits	Live via VReplication	N/A
Read replicas	Native, lag-aware routing	Native follower reads	Native follower reads	Native replication	Native, lag-aware	Native
Mature MySQL tooling	Yes	No	Partial	No	Yes	Yes
Sweet spot	Large MySQL workloads needing horizontal scale without rewriting SQL	Greenfield requiring strong consistency and global distribution	MySQL workloads on Kubernetes needing NewSQL guarantees	PostgreSQL with per-tenant or time-based sharding	Vitess without the ops burden	Small-medium single-server scale

The fundamental tradeoff is control versus simplicity. Vitess gives you a MySQL-compatible, sharded database that you can tune at every layer, but you own the VTTablet fleet, the topology service, and the VTGate fleet. CockroachDB and TiDB abstract the distributed layer away but trade MySQL wire compatibility or introduce new consistency models. Citus is the right answer if your data is already in PostgreSQL.

Closing

Vitess is not a drop-in replacement. It requires you to model your data around keyspaces and vindexes before you get the scaling benefit. Queries that ignore the sharding key become scatter operations, and cross-shard joins require rethinking access patterns. What Vitess gives you in return is MySQL semantics at a scale that a single MySQL server cannot reach, a live resharding path that does not require downtime, and a proven connection multiplexing layer that lets thousands of application pods share a bounded pool of database connections. The components are independently understandable, which means when something goes wrong, the failure domain is narrow enough to diagnose.