How Caching Works: Strategies, Patterns, and Pitfalls for Production Systems

Every senior engineer has said it: “We’ll add caching.” What follows is usually three months of subtle bugs, stale data incidents, and a cache that nobody trusts anymore. Caching is one of those areas where the concept is simple and the production reality is not.

This guide covers what you actually need to know: the core patterns, when each one applies, how to prevent stampedes, how to tune TTLs, and the failure modes that will find you if you don’t find them first. TypeScript examples throughout.

---

The Four Core Caching Patterns

Cache-Aside (Lazy Loading)

The most common pattern. Your application is responsible for loading data into the cache on demand.

import { Redis } from "ioredis";

const redis = new Redis();

async function getUserById(userId: string): Promise<User | null> {
  const cacheKey = `user:${userId}`;

  // 1. Check cache first
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached) as User;
  }

  // 2. Miss: load from database
  const user = await db.query<User>(
    "SELECT * FROM users WHERE id = $1",
    [userId]
  );

  if (!user) return null;

  // 3. Populate cache with TTL
  await redis.setex(cacheKey, 300, JSON.stringify(user)); // 5 min TTL

  return user;
}

The write path is straightforward: when you update a user, you invalidate or update the cache entry.

async function updateUser(userId: string, patch: Partial<User>): Promise<User> {
  const updated = await db.query<User>(
    "UPDATE users SET ... WHERE id = $1 RETURNING *",
    [userId]
  );

  // Invalidate: let the next read repopulate
  await redis.del(`user:${userId}`);

  return updated;
}

Cache-aside is resilient: if Redis goes down, your application falls back to the database. The cache is purely an optimization, not a dependency. The downside is cold start latency and the thundering herd problem when many requests hit a fresh cache simultaneously.

---

Read-Through

With read-through, the cache layer itself is responsible for loading from the database on a miss. Your application only ever talks to the cache.

// Conceptual: your cache client handles the miss
const cache = new ReadThroughCache({
  loader: async (key: string) => {
    const userId = key.replace("user:", "");
    return db.query<User>("SELECT * FROM users WHERE id = $1", [userId]);
  },
  ttl: 300,
});

async function getUserById(userId: string): Promise<User | null> {
  return cache.get(`user:${userId}`);
}

In practice this is often implemented with a decorator or a library like Cacheable or cache-manager in Node.js. The key difference from cache-aside: your application logic is simpler, but the cache becomes a harder dependency. If the cache layer fails, you need explicit fallback logic.

Read-through shines for read-heavy workloads with a well-defined key space, like product catalog pages or user profiles.

---

Write-Through

Every write goes to the cache and the database simultaneously. Reads always hit a warm cache.

async function updateUser(userId: string, patch: Partial<User>): Promise<User> {
  // Write to DB first
  const updated = await db.query<User>(
    "UPDATE users SET ... WHERE id = $1 RETURNING *",
    [userId]
  );

  // Write to cache immediately (no TTL or a long one)
  await redis.setex(`user:${userId}`, 3600, JSON.stringify(updated));

  return updated;
}

Write-through gives you strong consistency between cache and database at the cost of write latency. Every write now pays for two round trips. For write-heavy workloads this adds up fast.

It works best when reads vastly outnumber writes and stale reads are unacceptable. Session data, user preferences, and access control lists are reasonable candidates.

---

Write-Behind (Write-Back)

Writes go to the cache first and are flushed to the database asynchronously in batches. This is the fastest write pattern and the most dangerous.

class WriteBehindCache {
  private pendingWrites = new Map<string, { data: unknown; timestamp: number }>();
  private flushInterval: NodeJS.Timer;

  constructor(private redis: Redis, private flushMs = 1000) {
    this.flushInterval = setInterval(() => this.flush(), this.flushMs);
  }

  async set(key: string, data: unknown): Promise<void> {
    // Write to cache immediately
    await this.redis.set(key, JSON.stringify(data));
    // Queue for DB write
    this.pendingWrites.set(key, { data, timestamp: Date.now() });
  }

  private async flush(): Promise<void> {
    if (this.pendingWrites.size === 0) return;

    const batch = new Map(this.pendingWrites);
    this.pendingWrites.clear();

    // Write batch to DB
    await Promise.all(
      Array.from(batch.entries()).map(([key, { data }]) =>
        persistToDatabase(key, data)
      )
    );
  }
}

If the cache node crashes before flushing, you lose writes. Write-behind is appropriate for metrics aggregation, analytics counters, and non-critical state where some loss is acceptable. It is not appropriate for financial transactions, orders, or anything requiring durability.

---

Cache Invalidation: The Hard Part

Phil Karlton’s famous observation that cache invalidation is one of the two hardest problems in computer science is not a joke. Here are the main approaches.

TTL-based expiration is the simplest. Set a reasonable TTL and accept that data will be stale for up to that window. Most applications can tolerate this.

Event-driven invalidation uses change data capture (CDC) or event publishing to invalidate cache entries when data changes.

// After writing to DB, publish an event
await eventBus.publish("user.updated", { userId, fields: Object.keys(patch) });

// Cache service subscribes and invalidates
eventBus.subscribe("user.updated", async ({ userId }) => {
  await redis.del(`user:${userId}`);
  // Also invalidate derived keys
  await redis.del(`user:${userId}:permissions`);
  await redis.del(`feed:${userId}`);
});

This works well but introduces coupling between your application and the cache invalidation logic. You need to track every cache key that depends on a given piece of data. Missing one gives you a hard-to-reproduce stale data bug.

Versioned keys sidestep invalidation by making stale entries unreachable rather than deleting them.

async function getCacheKey(userId: string): Promise<string> {
  // Version stored in a fast lookup
  const version = await redis.get(`user:${userId}:version`) ?? "1";
  return `user:${userId}:v${version}`;
}

async function invalidateUser(userId: string): Promise<void> {
  // Bump version; old cache entries become orphaned
  await redis.incr(`user:${userId}:version`);
  // Optionally schedule cleanup of old keys
}

This approach is safe and simple but orphaned entries consume memory until they expire naturally. It works best with explicit TTLs.

---

Cache Stampede Prevention

A cache stampede (also called a thundering herd) happens when a popular cache entry expires and hundreds of requests simultaneously find a miss and all hit the database at once. Under load this can take down a database.

Mutex locking ensures only one request populates the cache on a miss.

async function getWithMutex(
  redis: Redis,
  key: string,
  loader: () => Promise<unknown>,
  ttl: number
): Promise<unknown> {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const lockKey = `lock:${key}`;
  const lockAcquired = await redis.set(lockKey, "1", "NX", "EX", 10);

  if (lockAcquired) {
    // This instance won the lock: load and populate
    try {
      const data = await loader();
      await redis.setex(key, ttl, JSON.stringify(data));
      return data;
    } finally {
      await redis.del(lockKey);
    }
  } else {
    // Another instance is populating: wait briefly and retry
    await new Promise((resolve) => setTimeout(resolve, 50));
    return getWithMutex(redis, key, loader, ttl);
  }
}

The NX flag on SET makes lock acquisition atomic. The EX flag ensures the lock releases even if the holder crashes.

Probabilistic early expiration is a subtler approach: recompute the cache entry before it expires, with probability increasing as expiration approaches.

function shouldRefreshEarly(ttlRemaining: number, beta = 1): boolean {
  // XFetch algorithm: refresh probability increases as TTL falls
  const noise = -Math.log(Math.random()) * beta;
  return noise > ttlRemaining;
}

async function getWithEarlyRefresh(
  redis: Redis,
  key: string,
  loader: () => Promise<unknown>,
  ttl: number
): Promise<unknown> {
  const [[, cached], [, remaining]] = await redis
    .pipeline()
    .get(key)
    .ttl(key)
    .exec() as [[null, string | null], [null, number]];

  if (cached && !shouldRefreshEarly(remaining)) {
    return JSON.parse(cached);
  }

  // Refresh
  const data = await loader();
  await redis.setex(key, ttl, JSON.stringify(data));
  return data;
}

Probabilistic early expiration avoids the thundering herd without coordination overhead. A single random request refreshes the key ahead of expiration.

---

Distributed Caching: Redis vs. Memcached

Dimension	Redis	Memcached
Data structures	Strings, hashes, lists, sets, sorted sets, streams	Strings only
Persistence	Optional (RDB snapshots, AOF log)	None
Replication	Yes (primary-replica, Sentinel, Cluster)	Client-side sharding only
Lua scripting	Yes (atomic operations)	No
Memory efficiency	Slightly lower for simple KV	Slightly higher for simple KV
Throughput	~100k ops/sec per node	~100k+ ops/sec per node
Use when	You need data structures, pub/sub, atomic scripts	Pure high-throughput string cache

For most applications Redis is the right default. Memcached has its place in read-heavy, simple-key workloads where you need to squeeze out memory efficiency.

Redis Cluster shards data across multiple nodes automatically using consistent hashing on key slots (16384 slots total). When using Cluster, multi-key operations must operate on keys that hash to the same slot. Use hash tags for this.

// Hash tag: {userId} forces both keys to the same slot
const pipeline = redis.pipeline();
pipeline.get(`{user:${userId}}:profile`);
pipeline.get(`{user:${userId}}:preferences`);
const results = await pipeline.exec();

---

CDN Caching

CDN caching operates at the network edge, before requests reach your origin. It is the highest-leverage caching layer for public content.

The key decisions are:

• Cache-Control headers: max-age controls browser cache, s-maxage controls CDN cache, stale-while-revalidate allows serving stale content while revalidating in the background.
• Surrogate keys / Cache tags: Custom headers (Cloudflare uses Cache-Tag, Fastly uses Surrogate-Key) that let you invalidate groups of cached responses by tag.

// Express: set caching headers on a product page response
app.get("/products/:id", async (req, res) => {
  const product = await getProduct(req.params.id);

  res
    .set("Cache-Control", "public, s-maxage=300, stale-while-revalidate=60")
    .set("Cache-Tag", `product:${product.id} category:${product.categoryId}`)
    .json(product);
});

// When a product is updated, purge by tag via Cloudflare API
async function purgeProductFromCDN(productId: string): Promise<void> {
  await fetch(
    `https://api.cloudflare.com/client/v4/zones/${ZONE_ID}/purge_cache`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${CF_API_TOKEN}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ tags: [`product:${productId}`] }),
    }
  );
}

stale-while-revalidate is underused. It gives users a fast response from the stale cache while the CDN fetches a fresh version in the background, effectively hiding revalidation latency entirely.

---

TTL Tuning

Getting TTL right matters more than most engineers realize. Too short and you pay for constant cache misses. Too long and stale data becomes a support incident.

A few heuristics:

• Match TTL to data change frequency. User profile data that changes weekly can have an hour TTL. Real-time inventory should be 5-30 seconds or event-invalidated.
• Add jitter to prevent synchronized expiration. If 10,000 keys all get the same TTL, they all expire together.

function ttlWithJitter(baseTtl: number, jitterFactor = 0.1): number {
  const jitter = baseTtl * jitterFactor * Math.random();
  return Math.floor(baseTtl + jitter);
}

// Base TTL of 300s, up to 10% variation
await redis.setex(key, ttlWithJitter(300), value);

• Use sliding TTLs for session-like data. Reset the TTL on each access so active users stay warm.

async function getSession(sessionId: string): Promise<Session | null> {
  const key = `session:${sessionId}`;
  const data = await redis.get(key);

  if (data) {
    // Reset TTL on each access
    await redis.expire(key, 1800); // 30 min from last access
    return JSON.parse(data);
  }

  return null;
}

---

Caching Tradeoffs at a Glance

Pattern	Consistency	Write Latency	Read Latency	Complexity	Best For
Cache-aside	Eventual (TTL)	DB only	Miss: DB + cache write	Low	General purpose reads
Read-through	Eventual (TTL)	DB only	Miss: DB + cache write	Medium	Read-heavy, well-defined keys
Write-through	Strong	DB + cache	Always fast	Medium	Low write rate, high read rate
Write-behind	Weak (async)	Cache only	Always fast	High	Counters, analytics, high write rate
No-cache fallback	N/A	DB only	DB only	Low	Cache-aside resilience mode

---

Production Failure Modes

Cache poisoning. A bug writes corrupted data to a cache key. Every subsequent read returns garbage. The fix: validate data on read, not just on write. A schema validation step before returning cached data can catch this.

Negative caching omission. You cache a 200 OK response but not a 404. Requests for missing resources always go to the database. For high-traffic APIs, cache negative results too with a short TTL.

const NOT_FOUND = "__NOT_FOUND__";

async function getUser(userId: string): Promise<User | null> {
  const cached = await redis.get(`user:${userId}`);

  if (cached === NOT_FOUND) return null;
  if (cached) return JSON.parse(cached);

  const user = await db.findUser(userId);

  if (!user) {
    await redis.setex(`user:${userId}`, 60, NOT_FOUND); // Cache the miss
    return null;
  }

  await redis.setex(`user:${userId}`, 300, JSON.stringify(user));
  return user;
}

Memory pressure and eviction. Redis evicts keys based on its eviction policy (LRU, LFU, random, etc.) when it hits maxmemory. If you have not set maxmemory and a policy, Redis will refuse new writes when the host runs out of memory. Always configure both in production.

Cold start after deploy. Deploying a new version that changes cache key formats or data shapes causes a full cache miss storm. Plan for this: either migrate keys gradually, use versioned keys, or warm the cache during deployment.

Cascading invalidation. Deleting one entity triggers invalidation of dozens of derived cache keys. This can create a spike of database load. Spread invalidation with a small delay or queue it.

---

Consistency Tradeoffs

Caching inherently trades consistency for performance. The question is not whether to accept staleness but how much and for what data.

Strong consistency between cache and database is possible with write-through plus synchronous invalidation, but the added latency often cancels out the read benefits. For most web applications, eventual consistency with a short TTL is the right default. For financial data, inventory, and anything with user-visible correctness requirements, either use write-through or skip the cache and optimize the database query instead.

Read replicas with cache-aside is a common middle ground: write to the primary, serve reads from a replica plus cache. The replica lag is typically 10-50ms, which is often acceptable and far cheaper than write-through overhead.

---

Closing

Caching is a tool for managing latency and database load, not a solution to slow queries. Before adding a cache, profile the actual bottleneck. An unindexed query will still be slow after caching if the miss rate is high enough to matter.

When you do add a cache, pick the pattern that matches your consistency requirements and write characteristics, tune TTLs to your data’s change rate, add jitter, handle the stampede, and cache your negative results. The failure modes are predictable. Most of them have been hitting production systems since Memcached launched in 2003. None of them are new.