How Valkey Works Internally: Multi-Threaded I/O, RDMA Networking, and the Fork That Is Replacing Redis in Production

In March 2024, Redis Ltd. relicensed Redis under a dual-license combining the Redis Source Available License (RSALv2) and the Server Side Public License (SSPLv1). Neither is OSI-approved. The practical effect was that cloud providers and managed-service operators could no longer bundle Redis in their offerings without a commercial agreement. Within weeks, a coalition of engineers from AWS, Google, Oracle, Ericsson, and others forked Redis 7.2.4 under a BSD license and donated it to the Linux Foundation. That fork is Valkey.

What started as a license-compliance story quickly became an engineering story. The Valkey maintainers are not managing a frozen snapshot of Redis 7.2. They are actively diverging the architecture: adding a multi-threaded I/O pipeline that goes further than Redis’s own threading additions, prototyping RDMA transport for data-center deployments, and backfilling cluster management improvements that the Redis roadmap had deprioritized. This article covers those architectural changes in enough depth to make an informed deployment decision.

The Fork’s Baseline and Governance

Valkey forked at Redis 7.2.4, which means it inherited Redis’s event loop (ae), data structures (SDS strings, dict, listpack, ziplist, quicklist, skiplist, rax), persistence (RDB snapshots, AOF, hybrid persistence), cluster (16,384 hash slots, CRC16, gossip protocol), and RESP2/RESP3 protocol support.

The Linux Foundation governance structure matters operationally. Major contributors include AWS (ElastiCache and MemoryDB teams), Google (Memorystore team), Oracle, and Ericsson. This breadth of corporate backing means the project has both the engineering bandwidth and the institutional incentive to maintain long-term compatibility while diverging internally. AWS migrated ElastiCache’s default engine to Valkey 7.2 in late 2024 and MemoryDB to Valkey 8 in 2025, providing production-scale validation that no community fork could replicate.

Multi-Threaded I/O: How Valkey Diverges from Redis

Redis introduced I/O threading in version 6.0 as an optional feature (io-threads, io-threads-do-reads). The model separates socket reading and writing from command execution: I/O threads read client buffers from the network and write response buffers back, while the main thread executes all commands serially against the data structures. Command execution remains single-threaded.

Valkey 8 extended this model in a meaningful way. The key change is that Valkey decoupled I/O thread management from command dispatch at the client level. In Redis 6+, I/O threads are assigned per connection in a round-robin fashion at connection creation, and the main thread collects parsed commands from all I/O threads before entering a single command-execution loop. In Valkey 8, the I/O thread pipeline was redesigned to eliminate the synchronization barrier between the I/O and main threads that caused latency spikes under high connection counts.

The Valkey 8 threading model works as follows:

1. Each I/O thread owns a set of client connections and runs its own epoll (or kqueue) loop, reading incoming data and writing pending output independently.
2. Parsed commands are placed on a per-thread command queue.
3. The main thread drains all command queues before executing commands, but the drain is lock-free (ring buffer hand-off rather than mutex-guarded list traversal).
4. Command execution still happens on the main thread for all commands touching data structures, preserving single-threaded semantics for data access.

The critical tradeoff: Valkey’s threading model retains the simplicity and correctness guarantees of Redis’s single-threaded command execution. You do not get the horizontal throughput scaling that Dragonfly achieves by partitioning the keyspace across cores. What you do get is better network throughput utilization at high connection counts without the CPU contention spikes that plagued Redis 6 I/O threads under bursty workloads.

// Measuring the I/O thread benefit in practice:
// Connect to Valkey 8 with io-threads = 4 configured,
// then compare throughput at different connection counts.
import { createClient } from "redis"; // valkey-compatible client

const client = createClient({ url: "redis://localhost:6379" });
await client.connect();

// Valkey 8 with io-threads=4 sustains ~600K ops/sec on a
// 16-core box at 1000 concurrent connections, compared to
// ~180K ops/sec on Redis 7.2 with io-threads=1 (main thread only).
// The gain comes from parallel socket I/O, not parallel command execution.

const pipeline = client.multi();
for (let i = 0; i < 100; i++) {
  pipeline.set(`key:${i}`, `value:${i}`);
}
const results = await pipeline.exec();
console.log(`Pipeline executed: ${results.length} commands`);

await client.disconnect();

For workloads that are CPU-bound on command execution (large SORT, LRANGE on long lists, expensive Lua scripts), Valkey’s threading model offers no relief. Those workloads are still serialized on the main thread. For workloads that are network I/O-bound at high concurrency, the Valkey 8 threading improvements deliver measurable latency reduction.

RDMA Networking Support

RDMA (Remote Direct Memory Access) allows a network adapter to read and write host memory directly, bypassing the CPU and the kernel networking stack. For in-memory data stores, RDMA eliminates the TCP overhead that typically dominates latency at sub-100-microsecond operation times.

Valkey 8.1 introduced an experimental RDMA transport layer, contributed primarily by engineers from Alibaba Cloud. The implementation uses the libibverbs / rdmacm userspace libraries and targets InfiniBand and RoCEv2 (RDMA over Converged Ethernet) NICs, which are common in hyperscale data center deployments.

The RDMA transport in Valkey replaces the TCP socket path for client-server communication. It does not replace the gossip protocol between cluster nodes (which remains TCP), and it does not use RDMA for persistence operations (RDB/AOF still go through the kernel file I/O path). The scope is narrowly: application client to server communication for GET/SET and similar command traffic.

At the implementation level, the RDMA path works through registered memory regions: both client and server pre-register memory buffers with the RDMA NIC’s protection domain. Incoming command data lands directly in those registered buffers without a kernel copy. The server’s event loop detects completions via the completion queue (CQ) poll, exactly as it does for epoll events in the TCP path, but with the kernel entirely out of the data path.

Practical constraints worth knowing:

• RDMA requires both client and server to be RDMA-capable. Your application client needs an RDMA-aware library, not the standard TCP Redis client.
• RoCEv2 requires priority flow control (PFC) and explicit congestion notification (ECN) configured at the switch layer. Getting this wrong causes head-of-line blocking that defeats the latency benefit.
• The feature is marked experimental in Valkey 8.1. It is a real production path for organizations that already run RDMA infrastructure (financial services, HPC), not a general deployment recommendation.

For most production deployments, RDMA is irrelevant. The architecturally interesting thing is that Valkey’s contributor base (Alibaba, Ericsson, AWS) operates infrastructure where 50-microsecond TCP round-trips are a bottleneck. The RDMA work signals where the project’s internals are heading.

Cluster Improvements: Slot Migration and Shard Balancing

Redis Cluster’s slot migration protocol has a known operational rough edge: during a CLUSTER SETSLOT <slot> MIGRATING / IMPORTING migration sequence, the migrating shard has to handle redirections (ASK responses) for every key in the slot until the migration completes. For large slots with millions of keys, migrations can take minutes or hours, during which all reads and writes to those keys pay an extra round-trip per command.

Valkey 8 introduced two improvements to the slot migration path:

Async migration with client-transparent handover. Valkey’s cluster can now migrate slots without requiring a hard cutover at the end. The migration process streams keys from source to destination using a background fiber, and the cluster state transitions from MIGRATING to NORMAL when the last key has been transferred. During the transfer window, the source shard handles reads and transparently proxies writes to the destination, so the client never sees an ASK redirection. This is not a client-facing protocol change (RESP remains unchanged), but it eliminates the ASK round-trip latency for active keys mid-migration.

Automatic slot rebalancing. Redis Cluster requires external tooling (redis-cli --cluster rebalance or custom automation) to move slots when adding or removing nodes. Valkey 8 added a built-in rebalancing controller that runs in the cluster coordinator role, monitoring slot distribution and initiating migrations when the imbalance exceeds a configurable threshold. The controller uses the async migration path described above, so rebalancing happens as a background operation without manual intervention.

// Slot migration state inspection with Valkey:
import { createClient } from "redis";

const client = createClient({ url: "redis://localhost:6379" });
await client.connect();

// Check which slots are currently migrating
const clusterInfo = await client.sendCommand(["CLUSTER", "INFO"]);
console.log(clusterInfo);

// Inspect individual node slot assignments
const clusterNodes = await client.sendCommand(["CLUSTER", "NODES"]);
const lines = (clusterNodes as string).split("\n").filter(Boolean);

for (const line of lines) {
  const parts = line.split(" ");
  const nodeId = parts[0];
  const flags = parts[2];
  const slots = parts.slice(8).join(" ");

  // [<-] prefix on slot range means IMPORTING, [>-] means MIGRATING
  if (slots.includes("[")) {
    console.log(`Node ${nodeId} (${flags}): in-progress migration: ${slots}`);
  }
}

await client.disconnect();

RESP Protocol Compatibility

Valkey is wire-compatible with Redis at the protocol level. RESP2 and RESP3 are both supported, and there are no Valkey-specific extensions to the wire protocol in Valkey 8 that would break an existing Redis client. This is intentional: the project explicitly scopes protocol changes as out-of-scope for the foreseeable future because compatibility with the existing Redis client ecosystem is a primary value proposition.

The practical consequence is that any client library that works against Redis 7.2 works against Valkey 8 without modification. The ioredis, redis (node-redis), Jedis, go-redis, redis-py, and Lettuce clients all connect and operate correctly. There is no “Valkey client” to install or configure.

The one protocol-level area where Valkey has diverged is in the responses to the INFO command. Valkey 8’s INFO server section includes a valkey_version field alongside redis_version (which is kept for compatibility). Monitoring tooling that parses INFO server programmatically should be updated to prefer valkey_version when present.

// Detecting Valkey vs Redis at runtime:
import { createClient } from "redis";

const client = createClient({ url: "redis://localhost:6379" });
await client.connect();

const info = await client.info("server");
const lines = info.split("\r\n");

const serverInfo: Record<string, string> = {};
for (const line of lines) {
  if (line.includes(":")) {
    const [key, value] = line.split(":");
    serverInfo[key.trim()] = value.trim();
  }
}

const isValkey = "valkey_version" in serverInfo;
const version = serverInfo["valkey_version"] ?? serverInfo["redis_version"];

console.log(`Server: ${isValkey ? "Valkey" : "Redis"} ${version}`);

await client.disconnect();

Production Migration from Redis to Valkey

Migrating from Redis 7.2 to Valkey is operationally straightforward because Valkey forked at that version and maintains full compatibility. The migration risks are not protocol or data-format risks but operational ones: monitoring dashboards, alerting rules, and configuration management that hardcodes “redis” identifiers.

Step 1: Inventory your Redis configuration. The Valkey configuration file uses the same keys as redis.conf. maxmemory, maxmemory-policy, appendonly, save, cluster-enabled, all map directly. The new Valkey-specific keys (like io-threads tuning and rdma-port for RDMA mode) are additive.

Step 2: Update INFO parsing in monitoring. Prometheus’s redis_exporter supports Valkey starting from v1.60.0. Before that version, it reads redis_version from INFO server, which Valkey still populates for compatibility, so it works but does not expose the valkey_version field. Updating the exporter is recommended but not urgent.

Step 3: Validate RDB compatibility. Valkey 7.2 uses the same RDB format as Redis 7.2 (RDB version 11). A snapshot taken on Redis 7.2 loads cleanly in Valkey 7.2, and vice versa. RDB files produced by Valkey 8 use RDB version 12 for some features and are not backward-compatible with Redis 7.2.

Step 4: Plan the io-threads configuration. The default in Valkey 8 is io-threads 1, which matches Redis behavior. For high-throughput deployments, set io-threads to the number of physical cores minus two (leaving headroom for the main command thread and the background task thread). Do not set it higher than physical core count.

# valkey.conf for a 16-core production node
maxmemory 24gb
maxmemory-policy allkeys-lru
io-threads 12
io-threads-do-reads yes
appendonly yes
appendfsync everysec
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000

Step 5: For AWS deployments, consider the managed path. If you are running on AWS, ElastiCache Valkey and MemoryDB for Valkey handle engine version upgrades, Multi-AZ failover, and snapshot management. The operational overhead of self-managing Valkey cluster node upgrades is non-trivial at scale.

A note on Lua compatibility: Valkey 8 ships Lua 5.4 (upgraded from Lua 5.1 in Redis). Lua scripts using math.random seeding behavior, table.unpack (which was unpack in 5.1), or string.gmatch patterns that relied on 5.1 semantics may behave differently. Run your Lua scripts in a Valkey 8 staging environment before production cutover.

Production Considerations

Threading and latency percentiles. The I/O threading improvements in Valkey 8 reduce p99 and p999 latency under high connection counts. Under low connection counts (under 100 concurrent clients), there is no meaningful difference from Redis 7.2 with io-threads 1. Profile your connection count distribution before deciding whether io-threads > 1 benefits your workload.

Snapshot fork latency. Valkey inherited Redis’s fork() + copy-on-write snapshot mechanism. On large instances (64GB+) with high write rates, the fork still causes latency spikes as the OS copies page table entries. The mitigation is the same as Redis: disable Transparent Huge Pages (echo never > /sys/kernel/mm/transparent_hugepage/enabled), size maxmemory at 60-70% of physical RAM to leave room for CoW pages, and schedule RDB snapshots during off-peak hours.

Cluster slot count. Valkey uses the same 16,384 hash slot space as Redis Cluster. This is an immutable constant in the protocol (encoded in CLUSTER NODES output and in CRC16 mod arithmetic). There is no path to a larger slot space without a breaking protocol change, and the Valkey project has no plans for this.

ACL and keyspace notification compatibility. Valkey ACLs, keyspace notifications (notify-keyspace-events), and pub/sub semantics are identical to Redis 7.2. No changes needed if you use these features.

License. Valkey is BSD-licensed, not source-available. This is the foundational reason to migrate: no usage restrictions, no negotiation with Redis Ltd., no audit risk from the RSALv2/SSPL terms.

Tradeoffs Table

Dimension	Valkey 8	Redis 7.2 (RSALv2)	Dragonfly	KeyDB	Memcached
License	BSD (OSI-approved)	RSALv2 / SSPL (not OSI)	BSL 1.1 (source-available)	BSD	BSD
Threading model	I/O threads (main thread executes commands)	I/O threads (optional, same model)	Shared-nothing per-core, all cores execute	Lock-striped dict, active-active	Multi-threaded throughout
Data structures	Full Redis set (string, hash, list, set, zset, stream, HLL)	Full Redis set	Redis-compatible subset (no streams in early versions)	Full Redis set	Strings only (flat key-value)
Persistence	RDB + AOF + hybrid	RDB + AOF + hybrid	Side-buffer snapshot (fork-free), AOF	RDB + AOF	None (in-memory only)
Replication	Leader-replica (async, optional WAIT for semi-sync)	Leader-replica	Journaled sharded replication	Active-active multi-master	None built-in
Cluster	Native cluster with async slot migration + auto-rebalance	Native cluster (manual rebalance required)	Cluster mode (emulated, no native gossip)	Multi-master, no native cluster	No built-in clustering
RDMA support	Experimental (8.1+, InfiniBand + RoCEv2)	None	None	None	None
Protocol	RESP2 + RESP3	RESP2 + RESP3	RESP2 + Memcached binary	RESP2 + RESP3	Memcached text + binary
Drop-in Redis compatibility	Full (wire, RDB, config)	N/A (baseline)	High (most commands), some gaps	High	Low (different protocol)
Throughput ceiling (single node)	Higher than Redis 7.2 at high concurrency via I/O threading	I/O-thread-bounded at high concurrency	Scales with core count, highest single-node ceiling	2-5x Redis on multi-core via lock striping	Highest raw GET/SET throughput (no complex types)
Managed cloud options	AWS ElastiCache, MemoryDB, GCP Memorystore, Azure	Redis Enterprise, Elasticache legacy	Dragonfly Cloud	KeyDB Cloud	AWS ElastiCache Memcached, GCP Memorystore
Sweet spot	Existing Redis deployments needing open-source license, cloud managed, or cluster auto-rebalance	Existing Redis Enterprise customers, locked into Redis Ltd.	Maximum single-node throughput, latency-critical, greenfield	Active-active multi-master write scaling	Pure cache, maximum throughput, no persistence, no complex types

Choosing Between the Alternatives

Valkey and Dragonfly solve different problems. Dragonfly is a ground-up rewrite that partitions the keyspace across all cores using shared-nothing fibers, reaching multi-million QPS on a single node at the cost of a more complex transaction model and a BSL 1.1 license. Valkey is a direct fork that maintains Redis’s single-threaded command execution model (adding better I/O threading around it), which means simpler correctness reasoning and full behavioral compatibility with Redis 7.2, at the cost of a lower throughput ceiling per node.

KeyDB sits between them: it uses the Redis codebase with a lock-striped dict that allows multiple threads to execute commands concurrently. This gives 2-5x throughput over Redis on multi-core hardware without requiring keyspace partitioning, but the locking model introduces potential contention under skewed access patterns that Valkey’s main-thread model avoids entirely.

For teams currently running Redis 7.2 and needing to move off the RSALv2/SSPL license, Valkey is the lowest-friction path: replace the binary, adjust the INFO parsing in monitoring, and carry on. The data format is compatible, the protocol is identical, and the operational model is the same. The cluster auto-rebalancing and improved I/O threading are meaningful additions, not just a license swap.

The RDMA work is worth watching if you operate in a data center environment where you control the network fabric. It is not a production recommendation today, but it signals that the project is building toward the low-latency use cases that currently require specialized hardware-software co-design.