How to Design a Rate Limiter: From Interview Answer to Production Code

The system design interview answer for rate limiting is almost always correct, and almost always incomplete. Token bucket, sliding window, fixed window: you pick one, draw a Redis box, say “atomic operations prevent races,” and move on. That answer gets you through the interview. It does not get you through a production incident at 2 AM when your limiter is letting through five times the allowed traffic.

The gap between the interview answer and the deployed system is where real complexity lives. This guide covers both, starting with the algorithms and ending with the production concerns that most tutorials skip.

The Algorithms

There are four common approaches. Each one involves a real tradeoff, not just a theoretical one.

Fixed Window Counter

Divide time into fixed buckets (one-minute windows, for example). Keep a counter per user per bucket. Reject when the counter exceeds the limit.

async function isAllowed(userId: string, limit: number): Promise<boolean> {
  // bucket changes every 60 seconds regardless of when the user started
  const window = Math.floor(Date.now() / 60_000);
  const key = `rate:${userId}:${window}`;

  const count = await redis.incr(key);
  if (count === 1) {
    // set TTL on first write, not on every write (avoids resetting the clock)
    await redis.expire(key, 120);
  }

  return count <= limit;
}

This is fast and cheap. One INCR per request, O(1) memory per user.

The problem is the window boundary. A user with a limit of 100 requests per minute can send 100 at 11:59:58 and another 100 at 12:00:01. Both windows see a count of 100. But 200 requests arrived in three seconds. Whether that matters depends on what you’re protecting.

Use this when: you need simple billing counters and burst traffic at window boundaries is not a security concern.

Sliding Window Log

Store a sorted set of request timestamps per user. On each request, remove entries outside the window, add the current timestamp, and count what remains.

async function isAllowed(
  userId: string,
  limit: number,
  windowMs: number
): Promise<boolean> {
  const now = Date.now();
  const windowStart = now - windowMs;
  const key = `rate:log:${userId}`;

  // pipeline all writes into one round trip
  const pipe = redis.pipeline();
  pipe.zremrangebyscore(key, 0, windowStart);     // prune expired entries
  pipe.zadd(key, now, `${now}-${Math.random()}`); // add this request
  pipe.zcard(key);                                 // count active entries
  pipe.expire(key, Math.ceil(windowMs / 1000) + 1);

  const results = await pipe.exec();
  const count = results[2][1] as number;

  return count <= limit;
}

No burst window problem. Every check looks at exactly the last N milliseconds.

The cost is memory. Each request writes a timestamp entry. At 1,000 requests per second across 10,000 users, you are storing ten million entries at any given moment. For most APIs this is acceptable. For high-throughput ingestion endpoints, it is not.

Use this when: you need precise windowing and your request volume is manageable.

Sliding Window Counter

A hybrid. Keep two fixed-window counters (the current window and the previous one) and estimate the count by weighting the previous window based on how far you have overlapped it.

estimated = prev_count * (1 - elapsed_fraction) + current_count

If you are 30% into the current window, you take 70% of the previous window’s count and add the full current count.

async function isAllowed(userId: string, limit: number): Promise<boolean> {
  const windowMs = 60_000;
  const now = Date.now();
  const currentWindow = Math.floor(now / windowMs);
  const prevWindow = currentWindow - 1;
  const elapsedFraction = (now % windowMs) / windowMs;

  const currentKey = `rate:${userId}:${currentWindow}`;
  const prevKey = `rate:${userId}:${prevWindow}`;

  const [currentCount, prevCount] = await Promise.all([
    redis.get(currentKey).then((v) => parseInt(v ?? "0")),
    redis.get(prevKey).then((v) => parseInt(v ?? "0")),
  ]);

  const estimated = prevCount * (1 - elapsedFraction) + currentCount;

  if (estimated >= limit) {
    return false;
  }

  await redis.pipeline().incr(currentKey).expire(currentKey, 120).exec();
  return true;
}

The approximation error is small in practice. Cloudflare published analysis showing less than 0.1% error under realistic traffic patterns. You get near-sliding-window accuracy with O(1) storage per user.

The failure mode is subtle: this version is not atomic. If two requests arrive simultaneously and both read the same estimated count below the limit, both increment, and you overshoot by one. For most limits this is fine. For strict billing or security limits, you need the Lua script version shown in the end-to-end section below.

Use this when: you need a good balance of accuracy, memory, and simplicity.

Token Bucket

Rather than counting requests in a time window, imagine a bucket that fills with tokens at a fixed rate. Each request consumes one token. An empty bucket means rejection.

The key difference from window-based approaches: a user who has been quiet for a while accumulates tokens and can burst above the average rate.

async function isAllowed(
  userId: string,
  capacity: number,    // maximum tokens (controls burst size)
  refillRate: number   // tokens added per second (controls average rate)
): Promise<boolean> {
  const key = `rate:bucket:${userId}`;
  const now = Date.now() / 1000;

  // Lua is required here: tokens and last_refill must be read and written atomically
  const script = `
    local key = KEYS[1]
    local capacity = tonumber(ARGV[1])
    local refill_rate = tonumber(ARGV[2])
    local now = tonumber(ARGV[3])

    local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
    local tokens = tonumber(bucket[1]) or capacity
    local last_refill = tonumber(bucket[2]) or now

    local elapsed = now - last_refill
    tokens = math.min(capacity, tokens + elapsed * refill_rate)

    if tokens < 1 then
      return 0
    end

    tokens = tokens - 1
    redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
    -- TTL = time to fill from empty, with buffer
    redis.call('EXPIRE', key, math.ceil(capacity / refill_rate) + 10)
    return 1
  `;

  const result = await redis.eval(script, 1, key, capacity, refillRate, now);
  return result === 1;
}

Token buckets are intuitive for API clients. A developer with a limit of 100 requests per minute and a bucket capacity of 20 can fire 20 requests immediately, then refill at roughly 1.67 per second. They know they can burst; they just cannot burst indefinitely.

You pay for this flexibility with reasoning complexity. “How many requests per hour did this user make?” requires math. For billing systems where exact counts matter, window-based approaches are easier to audit.

Use this when: you want to allow short bursts while enforcing a long-term average rate, and burst size is a product decision rather than a security boundary.

Distributed Rate Limiting

A single-process rate limiter is a solved problem. The distributed version is where most implementations break.

With multiple application servers, each one holds local state. Server A sees 80 requests from a user. Server B sees 70. The user has sent 150. Neither server knows it.

Using Redis as a Shared Store

The standard approach: use Redis as a central counter. All application servers write to the same key. Redis’s single-threaded command execution makes each command atomic, which eliminates most race conditions.

The nuance is hash tags in Redis Cluster. Without them, two keys for the same user might land on different shards, and a Lua script that touches both keys will fail because Lua scripts must execute on a single shard.

// WRONG for Redis Cluster: these two keys may land on different shards
const currentKey = `rate:${userId}:${currentWindow}`;
const prevKey = `rate:${userId}:${prevWindow}`;

// CORRECT: hash tag forces both keys to the same shard
const currentKey = `rate:{${userId}}:${currentWindow}`;
const prevKey = `rate:{${userId}}:${prevWindow}`;

The {userId} part is the hash tag. Redis uses only the content inside the braces to determine shard placement, so all keys sharing the same hash tag land on the same shard.

The Race You Will Miss

Even with Redis, there is a window for races if you split the check and the write into separate commands.

// WRONG: another request can slip between the GET and the INCR
const count = await redis.get(key);
if (parseInt(count ?? "0") < limit) {
  await redis.incr(key); // count may now be wrong
  return true;
}
return false;

The correct pattern is to increment first, then check. INCR is atomic. The count you get back is the true count including your request.

// CORRECT: INCR is atomic, so count reflects the true state
const count = await redis.incr(key);
if (count === 1) {
  await redis.expire(key, windowSeconds);
}
return count <= limit;

For multi-key logic (sliding window counter, token bucket), use Lua scripts. A Lua script in Redis runs atomically as a single transaction. There is no way for another command to interleave with it.

Approximate Distributed Rate Limiting

At very high scale (millions of requests per second, large number of keys), Redis can become a bottleneck. The practical solution is to accept bounded inaccuracy in exchange for throughput.

Each application server maintains a local counter. Every N milliseconds or every M local requests, it syncs to Redis. The local limit is set slightly below the global limit, so even with drift, you rarely exceed the true cap by a meaningful amount.

This is what large API gateways do in practice. If you need this level of scale, you have the traffic data to tune the parameters and verify the error bounds.

Multi-Tenant SaaS Rate Limiting

Most rate limiting tutorials cover single-user APIs. Multi-tenant SaaS has additional dimensions.

Plan-Based Limits

Different plans have different limits. Hardcoding these in the rate limiter is a maintenance problem. The cleaner approach: store plan limits in your database and look them up at request time (with a cache).

interface PlanLimits {
  requestsPerMinute: number;
  requestsPerDay: number;
  burstCapacity: number;
}

async function getPlanLimits(tenantId: string): Promise<PlanLimits> {
  const cacheKey = `plan:${tenantId}`;
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  const tenant = await db.tenant.findUnique({ where: { id: tenantId } });
  const limits = PLAN_LIMITS[tenant.plan];

  // cache for 5 minutes (plan changes don't need to propagate instantly)
  await redis.setex(cacheKey, 300, JSON.stringify(limits));
  return limits;
}

Per-Endpoint Limits

A user should not exhaust their search quota to prevent them from creating records. Separate limits by endpoint category.

const ENDPOINT_LIMITS: Record<string, { multiplier: number }> = {
  "/api/search": { multiplier: 1 },
  "/api/export": { multiplier: 10 },  // expensive operation, counts as 10 requests
  "/api/webhooks": { multiplier: 0 }, // internal calls, not counted
};

async function checkLimits(req: Request, tenantId: string): Promise<boolean> {
  const limits = await getPlanLimits(tenantId);
  const endpointConfig = ENDPOINT_LIMITS[req.path] ?? { multiplier: 1 };

  if (endpointConfig.multiplier === 0) return true;

  const [minuteOk, dayOk, ipOk] = await Promise.all([
    isAllowed(`minute:{${tenantId}}:${req.path}`, limits.requestsPerMinute),
    isAllowed(`day:{${tenantId}}`, limits.requestsPerDay),
    isAllowed(`ip:{${req.ip}}`, 2_000), // hard cap per IP regardless of tenant
  ]);

  return minuteOk && dayOk && ipOk;
}

Running the checks in parallel keeps the overhead low. Three Redis round trips that execute concurrently add roughly the same latency as one.

Tenant Isolation vs. Shared Infrastructure

One tenant hammering your API should not degrade the rate limiter for everyone else. With Redis, this is mostly automatic since each tenant writes to different keys. The risk is Redis CPU load from a single tenant with extremely high request volume.

If one tenant accounts for a disproportionate share of your traffic, consider a separate Redis instance or namespace for them. This is operational overhead, but it prevents noisy neighbor problems in your limiting infrastructure.

Response Headers and Retry Guidance

Clients that do not know they are being rate limited will retry immediately, making the problem worse. The standard headers tell them what happened and when to try again.

async function rateLimitMiddleware(req: Request): Promise<Response> {
  const result = await rateLimiter(`user:${req.userId}`, {
    limit: 100,
    windowMs: 60_000,
  });

  const headers = {
    "X-RateLimit-Limit": String(result.limit),
    "X-RateLimit-Remaining": String(result.remaining),
    "X-RateLimit-Reset": String(Math.floor(result.resetAt / 1000)),
  };

  if (!result.allowed) {
    const retryAfter = Math.ceil((result.resetAt - Date.now()) / 1000);
    return new Response("Too Many Requests", {
      status: 429,
      headers: {
        ...headers,
        "Retry-After": String(retryAfter),
      },
    });
  }

  // attach headers to every response, not just 429s
  return nextHandler(req, headers);
}

Return rate limit headers on every response, not just 429s. A client that can see X-RateLimit-Remaining: 5 will slow down proactively rather than waiting for a rejection.

Failure Handling

What happens when Redis is unavailable?

Fail open: allow all requests. Keeps your API available but removes protection. Appropriate for rate limits that exist primarily for billing fairness, where downtime is worse than overage.

Fail closed: reject all requests. Protects infrastructure but degrades the user experience. Appropriate for security-critical limits (authentication attempts, for example) where the cost of abuse is higher than the cost of downtime.

Most production systems fail open for normal API limits and fail closed for authentication endpoints. Make this a configuration option so you can change behavior without a redeploy.

async function rateLimiterWithFallback(
  key: string,
  config: RateLimitConfig
): Promise<RateLimitResult> {
  try {
    return await rateLimiter(key, config);
  } catch (err) {
    // log and alert, but do not silently ignore
    metrics.increment("rate_limiter.redis_error");
    logger.error({ err, key }, "Rate limiter unavailable");

    return {
      allowed: config.failOpen ?? true,
      remaining: 0,
      resetAt: Date.now() + config.windowMs,
      limit: config.limit,
    };
  }
}

End-to-End: Atomic Sliding Window Counter

Here is a full, production-ready implementation using a Lua-based sliding window counter. This is the version we reach for at Let’s Build Solutions when building multi-tenant APIs that need accurate limits without the memory cost of a log.

import { Redis } from "ioredis";

const redis = new Redis(process.env.REDIS_URL!);

interface RateLimitConfig {
  limit: number;
  windowMs: number;
  failOpen?: boolean;
}

interface RateLimitResult {
  allowed: boolean;
  remaining: number;
  resetAt: number;
  limit: number;
}

// Atomic sliding window counter via Lua.
// Uses hash tags so both keys land on the same Redis Cluster shard.
const SLIDING_WINDOW_SCRIPT = `
  local cur_key  = KEYS[1]
  local prev_key = KEYS[2]
  local limit    = tonumber(ARGV[1])
  local elapsed  = tonumber(ARGV[2])  -- fraction of current window elapsed (0.0 to 1.0)
  local ttl      = tonumber(ARGV[3])  -- seconds to keep each key alive

  local prev_count = tonumber(redis.call('GET', prev_key) or 0)
  local cur_count  = tonumber(redis.call('GET', cur_key) or 0)

  -- weight the previous window by how much of it is still in scope
  local estimated = prev_count * (1 - elapsed) + cur_count

  if estimated >= limit then
    return {0, 0}
  end

  local new_count = redis.call('INCR', cur_key)
  if new_count == 1 then
    redis.call('EXPIRE', cur_key, ttl)
  end

  -- remaining is approximate because estimated is a float
  local remaining = math.max(0, math.floor(limit - estimated - 1))
  return {1, remaining}
`;

export async function rateLimiter(
  key: string,
  config: RateLimitConfig
): Promise<RateLimitResult> {
  const { limit, windowMs, failOpen = true } = config;
  const now = Date.now();
  const currentWindow = Math.floor(now / windowMs);
  const prevWindow = currentWindow - 1;
  const elapsedFraction = (now % windowMs) / windowMs;
  const windowSeconds = Math.ceil(windowMs / 1000);
  const resetAt = (currentWindow + 1) * windowMs;

  // hash tags ensure both keys land on the same Redis Cluster shard
  const currentKey = `rl:{${key}}:${currentWindow}`;
  const prevKey = `rl:{${key}}:${prevWindow}`;

  try {
    const result = (await redis.eval(
      SLIDING_WINDOW_SCRIPT,
      2,
      currentKey,
      prevKey,
      limit,
      elapsedFraction,
      windowSeconds * 2 // TTL is 2x window so previous window is readable
    )) as [number, number];

    return {
      allowed: result[0] === 1,
      remaining: result[1],
      resetAt,
      limit,
    };
  } catch (err) {
    console.error({ err, key }, "Rate limiter error");
    return {
      allowed: failOpen,
      remaining: 0,
      resetAt,
      limit,
    };
  }
}

A few things worth calling out:

The TTL is set to 2x the window size. The previous window’s key needs to stay alive long enough for requests at the very start of the next window to read it. A TTL equal to the window size creates a race at the boundary.

The INCR return value is ignored for the allowed check. The Lua script handles the decision atomically. The TypeScript code only reads the result.

The remaining count is an approximation because estimated is a float. This is fine for response headers but not for strict billing calculations. If you need exact counts, use the sliding window log instead.

Algorithm Comparison

Algorithm             Memory       Accuracy      Burst Handling   Complexity
Fixed Window          O(1)         Low            Poor             Low
Sliding Window Log    O(requests)  Exact          Good             Medium
Sliding Window Ctr    O(1)         ~99.9%         Good             Medium
Token Bucket          O(1)         Exact avg      Configurable     Medium

For most production APIs, the sliding window counter is the right default. It is accurate enough for all but the strictest billing requirements, and the O(1) memory means you can add rate limiting without worrying about memory scaling.

For APIs where burst is a product feature (background job processors, webhook delivery), token bucket gives you explicit control over burst size.

Avoid fixed window for anything security-sensitive. The boundary burst is predictable and exploitable.

Closing Thoughts

Rate limiting errors come in two categories: correctness errors (letting through more than allowed) and availability errors (blocking legitimate traffic). Most production incidents are the first kind, caused by read-modify-write races that seemed fine during testing and broke under concurrent load.

The algorithm matters less than the atomicity. A sliding window log with a race condition is worse than a fixed window counter implemented correctly. Get the atomic operations right first, then choose the algorithm that fits your accuracy and memory requirements.

Instrument your rate limiter from day one. Log every rejection, track the ratio of rejected to allowed requests per tenant, and alert when a tenant approaches their limit. You will catch misconfigured limits and unusual traffic patterns well before they become incidents.