Building Resilient API Clients in TypeScript: Rate Limit Handling, Adaptive Throttling, and Retry Coordination for Third-Party Integrations

Every production SaaS application hits third-party APIs under real load and eventually discovers that “just add a retry” is not a strategy. Stripe throttles on requests per second. OpenAI throttles on tokens per minute. GitHub throttles on requests per hour with burst allowances. Twilio has per-account and per-phone-number limits. Each one is different, each one returns different headers, and none of them care that your background job queue just woke up 200 workers simultaneously.

This guide covers building a production-grade API client layer that handles all of this coherently: reading rate limit signals from responses, proactive client-side throttling before you hit server limits, coordinating limits across distributed workers, circuit breaking for degraded upstream services, and monitoring so you can see problems before they become incidents.

The Problem With Naive Retry Logic

Most teams start with something like this:

async function callWithRetry<T>(fn: () => Promise<T>, maxAttempts = 3): Promise<T> {
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (attempt === maxAttempts) throw err;
      await sleep(1000 * attempt);
    }
  }
  throw new Error("unreachable");
}

This fails in several ways. It retries non-retryable errors (4xx validation failures). It ignores Retry-After headers, so retries hit the server while it’s still limiting you. It doesn’t coordinate across concurrent callers, so ten workers each retrying independently creates a thundering herd. And it has no backpressure: the queue keeps filling while downstream is saturated.

Reading Rate Limit Signals

Most APIs return limit information in response headers. The de facto standard uses:

• X-RateLimit-Limit: total requests allowed in the window
• X-RateLimit-Remaining: requests left in the current window
• X-RateLimit-Reset: Unix timestamp (or seconds until reset, depending on the API) when the window resets
• Retry-After: seconds to wait before retrying (sent on 429 responses)

OpenAI also sends x-ratelimit-remaining-tokens and x-ratelimit-reset-tokens for its token-based limits. GitHub sends X-RateLimit-Used. The headers are inconsistent across providers, so you need per-provider parsing.

interface RateLimitState {
  limit: number;
  remaining: number;
  resetAt: Date;
  retryAfterMs?: number;
}

function parseRateLimitHeaders(headers: Headers, provider: string): RateLimitState | null {
  // Standard headers
  const limit = headers.get("x-ratelimit-limit");
  const remaining = headers.get("x-ratelimit-remaining");
  const reset = headers.get("x-ratelimit-reset");
  const retryAfter = headers.get("retry-after");

  if (!limit || !remaining || !reset) return null;

  const resetValue = parseInt(reset, 10);
  // Some APIs return seconds-until-reset, others return Unix timestamps.
  // A value under 1e9 is almost certainly a delta, not a timestamp.
  const resetAt =
    resetValue < 1_000_000_000
      ? new Date(Date.now() + resetValue * 1000)
      : new Date(resetValue * 1000);

  return {
    limit: parseInt(limit, 10),
    remaining: parseInt(remaining, 10),
    resetAt,
    retryAfterMs: retryAfter ? parseRetryAfter(retryAfter) : undefined,
  };
}

function parseRetryAfter(value: string): number {
  const numeric = parseFloat(value);
  if (!isNaN(numeric)) return numeric * 1000;
  // HTTP-date format
  const date = new Date(value);
  return Math.max(0, date.getTime() - Date.now());
}

Client-Side Token Bucket

Waiting for a 429 and then backing off is reactive. You can do better by maintaining a token bucket on the client that mirrors the server’s allowance. When the bucket is empty, requests wait locally instead of consuming quota and getting rejected.

class TokenBucket {
  private tokens: number;
  private lastRefill: number;
  private readonly capacity: number;
  private readonly refillRate: number; // tokens per millisecond

  constructor(capacity: number, refillPerSecond: number) {
    this.capacity = capacity;
    this.tokens = capacity;
    this.refillRate = refillPerSecond / 1000;
    this.lastRefill = Date.now();
  }

  private refill(): void {
    const now = Date.now();
    const elapsed = now - this.lastRefill;
    const newTokens = elapsed * this.refillRate;
    this.tokens = Math.min(this.capacity, this.tokens + newTokens);
    this.lastRefill = now;
  }

  async consume(tokens = 1): Promise<void> {
    this.refill();

    if (this.tokens >= tokens) {
      this.tokens -= tokens;
      return;
    }

    // Calculate wait time until enough tokens are available
    const deficit = tokens - this.tokens;
    const waitMs = deficit / this.refillRate;
    await sleep(waitMs);
    this.tokens = 0;
    this.lastRefill = Date.now();
  }

  // Called after a successful response to sync bucket with server state
  syncFromHeaders(state: RateLimitState): void {
    this.tokens = Math.min(this.capacity, state.remaining);
    this.lastRefill = Date.now();
  }

  // Called after a 429 to drain the bucket and wait for reset
  drainUntil(resetAt: Date): void {
    this.tokens = 0;
    this.lastRefill = resetAt.getTime();
  }
}

For token-based limits (OpenAI charges by tokens consumed, not requests), the bucket should track token units rather than request counts. You estimate the cost before sending and consume that many tokens from the bucket.

Adaptive Throttling

The token bucket handles steady-state flow, but adaptive throttling adds a second layer: slow down proactively as you approach the limit, before you actually hit it. This is especially useful when multiple parts of your application share the same API quota.

The idea is to introduce a delay proportional to how close you are to exhaustion. When you have 50% of your quota remaining, no delay. At 20% remaining, add a small delay. At 5%, add a significant delay. The exact curve depends on your traffic patterns.

function computeAdaptiveDelay(remaining: number, limit: number): number {
  const fraction = remaining / limit;

  if (fraction > 0.5) return 0;
  if (fraction > 0.2) return 50;   // 50ms when 20-50% remaining
  if (fraction > 0.1) return 200;  // 200ms when 10-20% remaining
  if (fraction > 0.05) return 500; // 500ms when 5-10% remaining
  return 1000;                     // 1s when under 5%
}

This is intentionally simple. You can replace the step function with a smooth curve (delay = baseDelay * (1 - fraction) ** 2) once you have data on your traffic shape.

The Rate-Aware HTTP Client

Combining header parsing, token bucket, and adaptive throttling into a reusable wrapper:

interface RateLimitConfig {
  requestsPerSecond: number;
  burstCapacity: number;
  provider: string;
}

interface RequestOptions extends RequestInit {
  tokenCost?: number; // for token-based rate limits
}

class RateLimitedClient {
  private readonly bucket: TokenBucket;
  private lastKnownState: RateLimitState | null = null;
  private readonly baseUrl: string;
  private readonly config: RateLimitConfig;

  constructor(baseUrl: string, config: RateLimitConfig) {
    this.baseUrl = baseUrl;
    this.config = config;
    this.bucket = new TokenBucket(config.burstCapacity, config.requestsPerSecond);
  }

  async fetch(path: string, options: RequestOptions = {}): Promise<Response> {
    const { tokenCost = 1, ...fetchOptions } = options;

    // Proactive: wait for bucket capacity
    await this.bucket.consume(tokenCost);

    // Adaptive: add delay based on last known state
    if (this.lastKnownState) {
      const delay = computeAdaptiveDelay(
        this.lastKnownState.remaining,
        this.lastKnownState.limit
      );
      if (delay > 0) await sleep(delay);
    }

    const response = await fetch(`${this.baseUrl}${path}`, fetchOptions);

    // Update local state from response headers
    const limitState = parseRateLimitHeaders(response.headers, this.config.provider);
    if (limitState) {
      this.lastKnownState = limitState;
      this.bucket.syncFromHeaders(limitState);
    }

    if (response.status === 429) {
      const retryAfterMs = limitState?.retryAfterMs ?? 5000;
      this.bucket.drainUntil(new Date(Date.now() + retryAfterMs));
      await sleep(retryAfterMs);
      return this.fetch(path, options); // single retry after 429
    }

    return response;
  }
}

Note: this single retry on 429 is intentional. If you hit a 429 after the bucket was already managing flow, something is wrong: either the bucket is misconfigured or there’s a distributed coordination problem. One retry surfaces that; unlimited retries hide it.

Coordinating Rate Limits Across Distributed Workers

The token bucket above is in-process. If you have ten workers sharing a single API key, each maintains its own independent bucket. The total throughput becomes 10x your actual limit, and you get systematic 429s.

The fix is a shared counter in Redis. The simplest approach uses a sliding window counter:

import { createClient } from "redis";

class RedisRateLimiter {
  private readonly redis: ReturnType<typeof createClient>;
  private readonly key: string;
  private readonly windowMs: number;
  private readonly limit: number;

  constructor(
    redis: ReturnType<typeof createClient>,
    key: string,
    limit: number,
    windowMs: number
  ) {
    this.redis = redis;
    this.key = key;
    this.limit = limit;
    this.windowMs = windowMs;
  }

  // Returns: { allowed: boolean; remaining: number; resetAt: Date }
  async checkAndConsume(cost = 1): Promise<{
    allowed: boolean;
    remaining: number;
    resetAt: Date;
  }> {
    const now = Date.now();
    const windowStart = now - this.windowMs;
    const windowKey = `ratelimit:${this.key}`;

    // Lua script for atomic check-and-increment
    const script = `
      local key = KEYS[1]
      local now = tonumber(ARGV[1])
      local window_start = tonumber(ARGV[2])
      local limit = tonumber(ARGV[3])
      local cost = tonumber(ARGV[4])
      local window_ms = tonumber(ARGV[5])

      -- Remove expired entries
      redis.call('ZREMRANGEBYSCORE', key, '-inf', window_start)

      -- Count current usage
      local current = redis.call('ZCARD', key)

      if current + cost > limit then
        return {0, limit - current, window_start + window_ms}
      end

      -- Add new entries (one per cost unit, each with unique member)
      for i = 1, cost do
        redis.call('ZADD', key, now, now .. ':' .. i .. ':' .. math.random(1e9))
      end
      redis.call('PEXPIRE', key, window_ms)

      return {1, limit - current - cost, window_start + window_ms}
    `;

    const result = (await this.redis.eval(script, {
      keys: [windowKey],
      arguments: [
        now.toString(),
        windowStart.toString(),
        this.limit.toString(),
        cost.toString(),
        this.windowMs.toString(),
      ],
    })) as [number, number, number];

    return {
      allowed: result[0] === 1,
      remaining: result[1],
      resetAt: new Date(result[2]),
    };
  }
}

Usage in a worker:

async function callWithDistributedLimit(
  client: RateLimitedClient,
  limiter: RedisRateLimiter,
  path: string,
  options: RequestOptions = {}
): Promise<Response> {
  const check = await limiter.checkAndConsume(options.tokenCost ?? 1);

  if (!check.allowed) {
    const waitMs = check.resetAt.getTime() - Date.now();
    await sleep(waitMs + 50); // small buffer for clock skew
    return callWithDistributedLimit(client, limiter, path, options);
  }

  return client.fetch(path, options);
}

The Lua script is atomic at the Redis level, so multiple workers do not race. The ZSET members use the current timestamp as the score, which enables precise sliding window eviction. If you need lower latency, the fixed-window variant (a simple INCR with EXPIREAT) is faster but allows up to 2x the limit at window boundaries.

Circuit Breaking

Rate limiting handles steady-state throttling. Circuit breaking handles a different failure mode: an API that’s consistently returning errors or timing out. You do not want to keep hammering a degraded upstream, both because it makes recovery harder and because you’re burning retry quota.

A minimal circuit breaker with three states (closed, open, half-open):

type CircuitState = "closed" | "open" | "half-open";

interface CircuitBreakerConfig {
  failureThreshold: number;   // failures before opening
  successThreshold: number;   // successes in half-open before closing
  timeoutMs: number;          // how long to stay open before trying half-open
  windowMs: number;           // sliding window for failure counting
}

class CircuitBreaker {
  private state: CircuitState = "closed";
  private failures: number[] = []; // timestamps of recent failures
  private successesInHalfOpen = 0;
  private openedAt: number | null = null;
  private readonly config: CircuitBreakerConfig;

  constructor(config: CircuitBreakerConfig) {
    this.config = config;
  }

  isOpen(): boolean {
    if (this.state === "closed") return false;

    if (this.state === "open") {
      const elapsed = Date.now() - (this.openedAt ?? 0);
      if (elapsed >= this.config.timeoutMs) {
        this.state = "half-open";
        this.successesInHalfOpen = 0;
        return false;
      }
      return true;
    }

    // half-open: allow requests through for probing
    return false;
  }

  recordSuccess(): void {
    if (this.state === "half-open") {
      this.successesInHalfOpen++;
      if (this.successesInHalfOpen >= this.config.successThreshold) {
        this.state = "closed";
        this.failures = [];
      }
    }
  }

  recordFailure(): void {
    const now = Date.now();
    this.failures = this.failures.filter(
      (t) => now - t < this.config.windowMs
    );
    this.failures.push(now);

    if (this.failures.length >= this.config.failureThreshold) {
      this.state = "open";
      this.openedAt = now;
    }
  }

  getState(): CircuitState {
    return this.state;
  }
}

Wire it into the client:

class ResilientClient {
  private readonly rateLimitedClient: RateLimitedClient;
  private readonly circuitBreaker: CircuitBreaker;

  constructor(rateLimitedClient: RateLimitedClient, circuitBreaker: CircuitBreaker) {
    this.rateLimitedClient = rateLimitedClient;
    this.circuitBreaker = circuitBreaker;
  }

  async fetch(path: string, options: RequestOptions = {}): Promise<Response> {
    if (this.circuitBreaker.isOpen()) {
      throw new Error(`Circuit open for ${path} — upstream is degraded`);
    }

    try {
      const response = await this.rateLimitedClient.fetch(path, options);

      if (response.status >= 500) {
        this.circuitBreaker.recordFailure();
      } else {
        this.circuitBreaker.recordSuccess();
      }

      return response;
    } catch (err) {
      this.circuitBreaker.recordFailure();
      throw err;
    }
  }
}

429s should not increment failure count. A 429 means the API is healthy and enforcing limits as designed. Only 5xx responses and network errors represent actual service degradation.

Tradeoffs

Approach	Latency overhead	Coordination complexity	Accuracy
Reactive retry on 429	None until failure	None	Wastes quota on rejected requests
In-process token bucket	Minimal	None	Good for single-process; fails under distributed load
Redis sliding window	1-5ms per request	Requires Redis	Accurate across workers; Redis becomes a dependency
Redis fixed window	<1ms per request	Requires Redis	Simpler, allows up to 2x limit at boundaries
Adaptive throttling	0-1000ms added	None (local)	Reduces 429s; delays are heuristic, not exact
Circuit breaker	None	None	Protects against cascade; needs tuning per API

The right combination depends on your deployment. A single Node.js process calling Stripe at low volume needs only the token bucket and reactive retry. A distributed background job system with 50 workers all sharing one GitHub token needs the Redis coordinator. APIs with inconsistent performance (external LLM providers, for example) benefit from circuit breaking.

Handling Fundamentally Different Rate Limit Schemes

Request-based limits (GitHub, Stripe): the token bucket works directly. Each request costs one token. The window size maps to the refill period.

Token-based limits (OpenAI): estimate the token cost from your request payload before sending, consume that from the bucket, and recalibrate after the response using actual token counts from the response body. Over-estimate when you’re unsure.

function estimateOpenAITokenCost(messages: Array<{ content: string }>): number {
  // Rough estimate: 1 token per 4 characters, plus overhead
  const charCount = messages.reduce((sum, m) => sum + m.content.length, 0);
  return Math.ceil(charCount / 4) + messages.length * 4 + 50;
}

Burst + sustained limits (some Twilio endpoints): two separate buckets, one for burst (short window, high limit) and one for sustained (long window, lower rate). A request must pass both before sending.

Per-resource limits (GitHub per-repository limits, Twilio per-phone-number limits): scope the Redis key by resource identifier, not just by API key. ratelimit:github:repo:org/name rather than ratelimit:github:global.

Production Considerations

Jitter on retry delays: when many workers hit a limit simultaneously and all sleep for the same duration, they wake up and hit the limit again together. Add jitter: retryAfterMs + Math.random() * 1000.

Observability: emit a metric every time you hit adaptive throttle, every 429, every circuit open event, and current bucket utilization. Without metrics, you won’t know whether your client-side throttling is working or whether you’re silently dropping load.

interface RateLimitMetrics {
  provider: string;
  event: "throttled" | "rate_limited" | "circuit_open" | "bucket_utilization";
  value?: number;
  path?: string;
}

// Emit to your metrics backend (Datadog, Prometheus, etc.)
declare function emitMetric(metric: RateLimitMetrics): void;

Separate limit budgets by priority: background sync jobs and user-facing requests sharing the same rate limit pool means a bulk job can starve interactive requests. Use separate API keys if the provider allows it, or implement priority queues with separate Redis counters: user-facing requests draw from a reserved high-priority pool, background jobs get whatever remains.

Propagate errors correctly: when the circuit is open or the rate limit is exhausted, throw a typed error that upstream callers can distinguish from a transient network error. This lets request handlers return appropriate HTTP 503 or 429 responses to their own callers rather than generic 500s.

class CircuitOpenError extends Error {
  readonly provider: string;
  constructor(provider: string) {
    super(`Circuit breaker open for ${provider}`);
    this.provider = provider;
  }
}

class RateLimitExhaustedError extends Error {
  readonly retryAfterMs: number;
  constructor(retryAfterMs: number) {
    super(`Rate limit exhausted, retry after ${retryAfterMs}ms`);
    this.retryAfterMs = retryAfterMs;
  }
}

Test with a real rate limit simulator: unit tests with mocked responses won’t catch coordination bugs. Write an integration test that spins up multiple workers against a test endpoint that enforces a low limit (10 requests per second) and verifies that aggregate throughput stays within bounds.

Clock skew: when comparing Date.now() against a Retry-After value from an upstream server, add 100-200ms as a buffer. Server and client clocks are not synchronized, and boundary conditions cause spurious retry failures.

The client-side bucket, the distributed Redis limiter, and the circuit breaker each solve a different failure mode. Used together, they make third-party API integrations behave predictably under load, instead of becoming the unpredictable source of cascading failures they so often are.