Designing an API Gateway: Routing, Authentication, and Rate Limiting at the Edge

An API gateway is one of those components that every distributed system eventually needs, and almost every team designs badly the first time. The failure modes are predictable: you either bolt on a hosted gateway without understanding what it actually does, so you can’t debug it when traffic spikes, or you build your own and discover three months later that you forgot about request timeouts, didn’t handle partial failures, and have no visibility into what’s happening at the edge.

This article covers the gateway as an orchestration layer. Not how to configure nginx, not which SaaS product to pick, but the actual design decisions: what a gateway owns, what it delegates, which architectural pattern fits your load, and what you need to get right before traffic exposes your gaps. The rate limiter internals are covered in a separate article on rate limiter design. Circuit breaking patterns are covered in the circuit breaker article. This article is about the gateway as the orchestration layer that ties those concerns together.

What a Gateway Actually Does

A gateway sits at the edge of your system and takes on responsibilities that would otherwise be duplicated across every upstream service. The canonical list:

• Routing: Map incoming requests to the correct upstream service, potentially rewriting paths, methods, or headers.
• Authentication and authorization: Verify identity before the request reaches any service. Ideally, services never see unauthenticated requests.
• Rate limiting: Enforce per-client, per-endpoint, or per-tenant quotas. This belongs at the edge so that abusive traffic is rejected before it consumes upstream resources.
• Request transformation: Rewrite headers, reshape payloads, inject tracing IDs, strip internal fields before responses leave the system.
• Observability: Emit structured logs and metrics for every request. This is the one place where you have a complete view of all traffic.
• Circuit breaking: Detect failing upstreams and stop forwarding traffic before cascading failures occur. Covered in depth in the circuit breaker article, but the gateway is where the breaker logic typically lives.

Each of these could be its own service. In practice, they compose naturally because they all operate on the same request lifecycle: receive, validate, enrich, forward, transform response, emit telemetry.

Architectural Patterns

Centralized Gateway

One gateway for all clients and all upstream services. Simple operationally, easy to reason about. The problem: every client has different needs. Mobile clients want compact payloads and fewer round trips. Internal services want low latency and full data. Browser clients need CORS headers and cookie-based auth. A centralized gateway either becomes a negotiation surface for all those concerns, or it over-serves some clients and under-serves others.

Works well when: you have a small number of clients with similar needs, or when your upstreams are simple enough that client-specific logic isn’t necessary yet.

Backend for Frontend (BFF)

Each client type gets its own gateway that speaks the client’s language. The mobile BFF aggregates calls to three microservices into one response. The web BFF handles session cookies and renders server-side data. The internal BFF skips auth overhead for service-to-service calls.

The tradeoff is operational: you now have N gateways to maintain. Cross-cutting concerns (rate limiting, tracing) need to be consistent across all of them, which means shared middleware or a base library that all BFFs import.

Works well when: clients have meaningfully different data shapes or performance requirements, or when you’re dealing with a mobile app that needs to minimize requests over expensive connections.

Edge Gateway

Run the gateway at the network edge, close to users, in multiple regions. This reduces latency for request validation (auth token verification, rate limit checks) and allows you to terminate bad traffic before it reaches your origin infrastructure.

The constraint: edge environments have limited compute and storage. You can’t run a full Redis client at the edge in most platforms. Rate limiting state either needs to be approximate (using local counters that sync periodically) or you need a globally replicated store. Auth verification needs to be stateless (JWT verification works; session lookup does not without a network hop).

Core Implementation

Let’s build a minimal but realistic gateway in TypeScript. The goal is to show the structure, not a production-complete implementation.

Request Context

Every gateway operation works on a shared context object that accumulates state as the request moves through the pipeline.

interface GatewayContext {
  requestId: string;
  startedAt: number;
  request: Request;
  upstream: UpstreamConfig | null;
  identity: AuthIdentity | null;
  rateLimitResult: RateLimitResult | null;
  response: Response | null;
  error: GatewayError | null;
}

interface AuthIdentity {
  tenantId: string;
  userId: string;
  scopes: string[];
}

interface RateLimitResult {
  allowed: boolean;
  remaining: number;
  resetAt: number;
  limit: number;
}

Middleware Pipeline

A gateway is a pipeline. Each stage can short-circuit by writing a response to the context. Later stages check whether the context already has a response and skip their logic if so.

type Middleware = (ctx: GatewayContext, next: () => Promise<void>) => Promise<void>;

async function runPipeline(
  ctx: GatewayContext,
  middlewares: Middleware[]
): Promise<void> {
  let index = 0;

  async function next(): Promise<void> {
    if (index >= middlewares.length) return;
    const middleware = middlewares[index++];
    await middleware(ctx, next);
  }

  await next();
}

Routing

Routing matches an incoming request to an upstream configuration. A realistic router handles path parameters, prefix matching, and method constraints.

interface UpstreamConfig {
  id: string;
  baseUrl: string;
  pathRewrite?: (path: string) => string;
  timeout: number;
  retries: number;
  circuitBreaker: CircuitBreakerConfig;
}

interface RouteConfig {
  method: string | string[];
  pathPattern: string; // e.g., "/api/v1/users/:id"
  upstream: UpstreamConfig;
  authRequired: boolean;
  rateLimit?: RateLimitConfig;
}

function matchRoute(
  routes: RouteConfig[],
  method: string,
  path: string
): { route: RouteConfig; params: Record<string, string> } | null {
  for (const route of routes) {
    const methods = Array.isArray(route.method) ? route.method : [route.method];
    if (!methods.includes(method) && !methods.includes("*")) continue;

    const result = matchPath(route.pathPattern, path);
    if (result) return { route, params: result.params };
  }
  return null;
}

const routeMiddleware: Middleware = async (ctx, next) => {
  const url = new URL(ctx.request.url);
  const matched = matchRoute(routes, ctx.request.method, url.pathname);

  if (!matched) {
    ctx.response = new Response(JSON.stringify({ error: "not_found" }), {
      status: 404,
      headers: { "Content-Type": "application/json" },
    });
    return;
  }

  ctx.upstream = matched.route.upstream;
  await next();
};

Authentication

The gateway verifies tokens before forwarding requests. For JWTs, verification is stateless: validate the signature, check expiry, extract claims. For opaque tokens, you need a lookup, which means a cache with a TTL shorter than your token expiry window.

const authMiddleware: Middleware = async (ctx, next) => {
  const route = ctx.upstream; // already set by routing middleware
  if (!requiresAuth(ctx.request.url.pathname)) {
    await next();
    return;
  }

  const authHeader = ctx.request.headers.get("Authorization");
  if (!authHeader?.startsWith("Bearer ")) {
    ctx.response = unauthorized("missing_token");
    return;
  }

  const token = authHeader.slice(7);

  try {
    const identity = await verifyJWT(token, {
      issuer: "https://auth.example.com",
      audience: "api.example.com",
      algorithms: ["RS256"],
    });
    ctx.identity = identity;
    await next();
  } catch (err) {
    if (err instanceof TokenExpiredError) {
      ctx.response = unauthorized("token_expired");
    } else if (err instanceof InvalidTokenError) {
      ctx.response = unauthorized("invalid_token");
    } else {
      // Don't leak internal errors through auth failures
      ctx.response = unauthorized("auth_failed");
      logError("auth_verification_error", err);
    }
  }
};

function unauthorized(code: string): Response {
  return new Response(JSON.stringify({ error: code }), {
    status: 401,
    headers: {
      "Content-Type": "application/json",
      "WWW-Authenticate": 'Bearer error="invalid_token"',
    },
  });
}

One important detail: do not leak error specifics through auth failures. An attacker probing your gateway should not be able to distinguish between “this token is expired” and “this token was never valid.” Both return 401. Log the specifics internally.

Rate Limiting

The gateway applies rate limits using the identity established during auth. Per-tenant and per-user limits are the most common shapes. The implementation details live in the rate limiter article; at the gateway layer, the concern is how to wire the result into the pipeline.

const rateLimitMiddleware: Middleware = async (ctx, next) => {
  if (!ctx.identity) {
    await next();
    return;
  }

  const key = `rl:tenant:${ctx.identity.tenantId}`;
  const result = await checkRateLimit(key, {
    limit: getTenantLimit(ctx.identity.tenantId),
    windowSeconds: 60,
  });

  ctx.rateLimitResult = result;

  // Always add headers, even when not limited
  // Clients need this to implement backoff
  const headers = {
    "X-RateLimit-Limit": String(result.limit),
    "X-RateLimit-Remaining": String(result.remaining),
    "X-RateLimit-Reset": String(result.resetAt),
  };

  if (!result.allowed) {
    ctx.response = new Response(
      JSON.stringify({ error: "rate_limit_exceeded" }),
      {
        status: 429,
        headers: {
          ...headers,
          "Content-Type": "application/json",
          "Retry-After": String(Math.ceil((result.resetAt - Date.now()) / 1000)),
        },
      }
    );
    return;
  }

  await next();
};

Always emit the rate limit headers, even when the request is allowed. Clients use X-RateLimit-Remaining to implement proactive backoff. If they only see headers on 429 responses, they have no signal until they’re already throttled.

Upstream Forwarding

The forwarding step constructs the upstream request, applies path rewrites, injects internal headers (trace ID, authenticated identity claims), and handles the response.

const forwardMiddleware: Middleware = async (ctx, next) => {
  if (ctx.response) return; // already handled upstream

  const upstream = ctx.upstream!;
  const url = new URL(ctx.request.url);
  const upstreamPath = upstream.pathRewrite
    ? upstream.pathRewrite(url.pathname)
    : url.pathname;

  const upstreamUrl = `${upstream.baseUrl}${upstreamPath}${url.search}`;

  const requestHeaders = new Headers(ctx.request.headers);
  requestHeaders.set("X-Request-Id", ctx.requestId);
  requestHeaders.set("X-Forwarded-For", getClientIp(ctx.request));

  if (ctx.identity) {
    // Forward identity as signed headers, not raw values
    // Services trust the gateway, not the client
    requestHeaders.set("X-Tenant-Id", ctx.identity.tenantId);
    requestHeaders.set("X-User-Id", ctx.identity.userId);
    requestHeaders.set("X-User-Scopes", ctx.identity.scopes.join(","));
    // Remove the raw auth token — services don't need it
    requestHeaders.delete("Authorization");
  }

  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), upstream.timeout);

  try {
    const upstreamResponse = await fetch(upstreamUrl, {
      method: ctx.request.method,
      headers: requestHeaders,
      body: ctx.request.body,
      signal: controller.signal,
    });

    ctx.response = upstreamResponse;
    await next();
  } catch (err) {
    if (err instanceof DOMException && err.name === "AbortError") {
      ctx.response = gatewayError(504, "upstream_timeout");
    } else {
      ctx.response = gatewayError(502, "upstream_unavailable");
    }
  } finally {
    clearTimeout(timeout);
  }
};

The identity forwarding pattern is worth calling out: the gateway strips the raw Authorization header and replaces it with internal headers that services trust. This means services don’t need to verify JWTs themselves. They trust the gateway to have done it. This only works if services are not publicly accessible and if the gateway-to-service channel is secure.

Tradeoffs

Concern	Centralized Gateway	BFF Pattern	Edge Gateway
Operational overhead	Low (one service)	High (N services)	Medium (one service, complex deploy)
Client optimization	Hard (serves all clients equally)	Strong (each BFF owns its client)	Limited (edge compute constraints)
Latency	Medium (one hop, single region)	Medium (one hop, single region)	Low (geographically close to clients)
Cross-cutting consistency	Easy (one place to change)	Hard (shared lib or copy-paste)	Medium (centralized config, distributed execution)
Auth at edge	Possible with stateless JWT	Same	Requires stateless auth only
Rate limit consistency	Strong (single counter store)	Weak (separate stores)	Approximate (no strong distributed state)
Failure blast radius	High (one gateway fails, everything fails)	Low (one BFF fails, other clients unaffected)	Low (regional failures, not global)
Debugging	Easy (single log stream)	Medium (N log streams, need correlation)	Medium (distributed logs, trace IDs required)

The right choice is not universal. A BFF makes sense when you have a mobile app with different data needs than your web app. An edge gateway makes sense when you have global users and auth latency matters. A centralized gateway is often the right starting point because it’s easy to operate and easy to change.

Production Considerations

Graceful Degradation

A gateway that fails hard on upstream errors is worse than no gateway. Define explicit fallback behaviors per route. Some upstreams are critical (auth service down means no requests can proceed). Others are optional (recommendation service down means serve a default response, not a 503).

interface UpstreamConfig {
  // ... previous fields
  fallback?: {
    type: "static" | "cache" | "empty";
    staticResponse?: object;
    cacheTtl?: number;
  };
}

async function forwardWithFallback(
  ctx: GatewayContext,
  upstream: UpstreamConfig
): Promise<Response> {
  try {
    return await forwardToUpstream(ctx, upstream);
  } catch (err) {
    if (!upstream.fallback) throw err;

    recordFallbackUsed(upstream.id);

    switch (upstream.fallback.type) {
      case "static":
        return new Response(JSON.stringify(upstream.fallback.staticResponse), {
          status: 200,
          headers: {
            "Content-Type": "application/json",
            "X-Gateway-Fallback": "static",
          },
        });
      case "cache":
        return getCachedResponse(ctx.request) ?? gatewayError(503, "upstream_down");
      case "empty":
        return new Response(JSON.stringify({}), { status: 200 });
    }
  }
}

Observability

The gateway is your best observability point. Every request passes through it. Emit structured logs with enough context to reconstruct what happened: request ID, tenant, upstream, duration, status codes (both gateway and upstream), rate limit state, circuit breaker state.

const observabilityMiddleware: Middleware = async (ctx, next) => {
  await next();

  const duration = Date.now() - ctx.startedAt;
  const status = ctx.response?.status ?? 0;

  structuredLog({
    event: "gateway_request",
    requestId: ctx.requestId,
    method: ctx.request.method,
    path: new URL(ctx.request.url).pathname,
    upstreamId: ctx.upstream?.id ?? "unrouted",
    tenantId: ctx.identity?.tenantId ?? "anonymous",
    statusCode: status,
    durationMs: duration,
    rateLimitRemaining: ctx.rateLimitResult?.remaining ?? null,
    error: ctx.error?.code ?? null,
  });

  // Emit metrics separately from logs
  incrementCounter("gateway.requests", {
    upstream: ctx.upstream?.id ?? "unrouted",
    status: String(status),
  });
  recordHistogram("gateway.request_duration_ms", duration, {
    upstream: ctx.upstream?.id ?? "unrouted",
  });
};

Configuration Management

Route configs and upstream configs change frequently. Hard-coding them means a deploy for every route change. The standard pattern is a config store (database or file-backed) with a watch mechanism that reloads routes without restarting the process.

For small systems, a YAML file watched with fs.watch is sufficient. For larger systems, you want a control plane that manages config centrally and pushes updates to gateway instances. This is the pattern behind most service mesh implementations.

Health Checks and Draining

The gateway needs to handle restarts gracefully. When a new instance starts, it should not receive traffic until it has loaded its configuration and established connections to its dependencies (Redis for rate limiting, config store). When shutting down, it should stop accepting new connections, finish in-flight requests, and then exit.

let isReady = false;
let isDraining = false;

// Kubernetes readiness probe
app.get("/healthz/ready", (req, res) => {
  if (!isReady || isDraining) {
    res.status(503).json({ status: "not_ready" });
  } else {
    res.status(200).json({ status: "ready" });
  }
});

process.on("SIGTERM", async () => {
  isDraining = true;
  // Give load balancer time to stop routing new requests
  await sleep(5000);
  await server.close();
  process.exit(0);
});

Build vs. Adopt

The honest answer: you probably should not build a full gateway from scratch. The middleware pipeline pattern above is useful for understanding, and for small-scale internal gateways where you need specific behavior. But production gateways need connection pooling, TLS termination, hot config reloads, distributed tracing integration, and battle-tested routing code that has handled edge cases you haven’t thought of yet.

The exception is when you’re running at the edge (Cloudflare Workers, Fastly Compute) where existing gateway software doesn’t run, or when your requirements are narrow enough that a 200-line middleware pipeline handles everything you need.

What the build-vs-adopt framing misses is that understanding the internals matters even when you’re using an off-the-shelf solution. You can’t tune what you don’t understand. You can’t debug a rate limiting anomaly if you don’t know whether your gateway is using fixed window or sliding window. You can’t design your auth flow correctly if you don’t understand how identity gets forwarded to upstreams. The code in this article is meant to make those concepts concrete, not to be copy-pasted into production.

The Gateway as a Contract

A well-designed gateway is a contract between your infrastructure and your clients. Clients know that if they have a valid token and stay within their rate limit, their requests will reach the right service. Services know that any request that reaches them has been authenticated and carries verified identity headers.

Designing that contract carefully, making it explicit in code, and making it observable under production load: that’s the work. The specific technology you pick to run it matters much less.