WebSocket Architecture at Scale: Connection Management, Heartbeats, and Horizontal Scaling

You deploy a WebSocket server. It works beautifully in development. Then you hit production and the problems arrive in sequence: connections drop silently, your second server instance can’t reach connections held by the first, memory climbs until the process restarts, and a slow client starts backing up your entire message queue.

None of these problems are exotic. They are the standard experience of running WebSockets at any meaningful scale. The protocol itself is simple. The operational surface is not.

This covers the full lifecycle: from the HTTP upgrade handshake through connection management, keepalive design, horizontal scaling strategies, backpressure, and reconnection. It also covers when to reach for SSE or long polling instead.

The Protocol Upgrade

WebSocket connections start as HTTP/1.1 requests. The client sends an Upgrade header, and if the server agrees, the TCP connection is repurposed. No new socket is opened. The HTTP connection is hijacked.

// What the browser sends
GET /ws HTTP/1.1
Host: api.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

// What a compliant server returns
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

The Sec-WebSocket-Accept value is derived from the client’s key plus a fixed GUID, then SHA-1 hashed and base64 encoded. This handshake proves the server actually understands the WebSocket protocol rather than blindly echoing headers.

After 101, the connection speaks WebSocket framing. Each message is a frame with an opcode (text, binary, ping, pong, close), a masking bit (client-to-server frames are always masked), and a payload. The framing is compact and the full spec fits in an afternoon.

In Node.js, you rarely implement this directly. But understanding what happens at the TCP layer matters when you’re debugging dropped connections or writing a proxy.

Connection Lifecycle Management

A WebSocket connection has four distinct states: connecting, open, closing, and closed. The gap between “connection is closed” and “application knows about it” is where most silent failures live.

import { WebSocketServer, WebSocket } from 'ws';

interface Connection {
  id: string;
  socket: WebSocket;
  userId: string;
  lastPong: number;
  subscriptions: Set<string>;
}

class ConnectionManager {
  private connections = new Map<string, Connection>();

  register(socket: WebSocket, userId: string): string {
    const id = crypto.randomUUID();
    const conn: Connection = {
      id,
      socket,
      userId,
      lastPong: Date.now(),
      subscriptions: new Set(),
    };

    this.connections.set(id, conn);

    socket.on('close', (code, reason) => {
      this.cleanup(id, code, reason.toString());
    });

    socket.on('error', (err) => {
      console.error({ connId: id, err }, 'socket error');
      this.cleanup(id, 1011, err.message);
    });

    return id;
  }

  private cleanup(id: string, code: number, reason: string): void {
    const conn = this.connections.get(id);
    if (!conn) return;

    // Unsubscribe from all channels before removing
    for (const channel of conn.subscriptions) {
      this.unsubscribe(id, channel);
    }

    this.connections.delete(id);
    console.info({ connId: id, code, reason }, 'connection closed');
  }

  getByUser(userId: string): Connection[] {
    return [...this.connections.values()].filter(c => c.userId === userId);
  }
}

One user can have multiple concurrent connections (multiple tabs, mobile app plus browser). Your ConnectionManager needs to track the user-to-connection mapping, not assume one-to-one.

Close codes matter. 1000 is a clean close. 1001 means the endpoint is going away (server restart, browser navigation). 1006 means the connection was closed abnormally without a close frame, which is the fingerprint of a network drop. Log close codes. They tell you whether your load balancer is aggressively timing out idle connections or whether clients are genuinely disconnecting.

Heartbeat Design

TCP keepalive exists, but it fires on a timer that defaults to hours on most operating systems and is often disabled by load balancers. You cannot rely on it to detect dead connections in a reasonable timeframe.

WebSocket defines ping and pong frames at the protocol level. The server sends a ping; the client is required to respond with a pong. If it doesn’t, the connection is gone.

const HEARTBEAT_INTERVAL_MS = 30_000;
const HEARTBEAT_TIMEOUT_MS = 10_000;

class HeartbeatManager {
  private timers = new Map<string, NodeJS.Timeout>();

  start(connId: string, conn: Connection, manager: ConnectionManager): void {
    const schedule = () => {
      const timer = setInterval(() => {
        if (conn.socket.readyState !== WebSocket.OPEN) {
          this.stop(connId);
          return;
        }

        const elapsed = Date.now() - conn.lastPong;
        if (elapsed > HEARTBEAT_INTERVAL_MS + HEARTBEAT_TIMEOUT_MS) {
          // Client missed a pong. Terminate, don't just close.
          console.warn({ connId }, 'heartbeat timeout, terminating');
          conn.socket.terminate();
          this.stop(connId);
          return;
        }

        conn.socket.ping();
      }, HEARTBEAT_INTERVAL_MS);

      this.timers.set(connId, timer);
    };

    conn.socket.on('pong', () => {
      conn.lastPong = Date.now();
    });

    schedule();
  }

  stop(connId: string): void {
    const timer = this.timers.get(connId);
    if (timer) {
      clearInterval(timer);
      this.timers.delete(connId);
    }
  }
}

The distinction between socket.close() and socket.terminate() matters here. close() initiates a graceful WebSocket closing handshake. terminate() destroys the underlying TCP socket immediately. For a heartbeat timeout, you want terminate(). The client is already gone. The closing handshake will never complete.

Set your heartbeat interval relative to what your load balancer and proxies will tolerate. AWS ALB has a 60-second idle timeout by default. Nginx’s proxy_read_timeout defaults to 60 seconds. Ping every 25-30 seconds to keep the connection alive through both.

Horizontal Scaling: The Core Problem

A single WebSocket server works fine until it doesn’t. When you add a second instance, you face a fundamental problem: a message intended for user A needs to reach the connection that holds user A, but that connection might be on any server in your fleet.

Sticky Sessions

The first instinct is sticky sessions at the load balancer: route a given client’s requests always to the same server. This solves the problem partially. A client maintains its connection to one server, and messages can be delivered without cross-server coordination.

The failure modes are significant. A server restart or crash loses all its connections simultaneously. Uneven connection distribution can leave one server overwhelmed while others are idle. And sticky sessions require session affinity based on a cookie or IP, which breaks in environments with CGNAT or shared proxies.

Sticky sessions are a starting point, not a solution.

Redis Pub/Sub Fan-out

The standard architecture for small-to-medium scale: each server instance subscribes to a Redis channel. When a message needs to be delivered to user A, publish it to Redis. Every server receives it and delivers it to any local connections for user A.

import { createClient } from 'redis';

const redis = createClient({ url: process.env.REDIS_URL });
const subscriber = redis.duplicate();

await redis.connect();
await subscriber.connect();

// Subscribe to the shared fan-out channel
await subscriber.subscribe('messages', (rawMessage) => {
  const message = JSON.parse(rawMessage) as {
    userId: string;
    payload: unknown;
  };

  // Find local connections for this user and deliver
  const conns = connectionManager.getByUser(message.userId);
  for (const conn of conns) {
    if (conn.socket.readyState === WebSocket.OPEN) {
      conn.socket.send(JSON.stringify(message.payload));
    }
  }
});

// To send a message from anywhere in the system:
async function sendToUser(userId: string, payload: unknown): Promise<void> {
  await redis.publish('messages', JSON.stringify({ userId, payload }));
}

Every server receives every message, even if it has no connections for that user. At high message volume, this becomes wasteful. A more targeted approach uses per-server channels: each server subscribes to its own channel, and a routing layer looks up which server holds a given user’s connection.

const SERVER_ID = process.env.POD_NAME ?? crypto.randomUUID();

// Register connections in Redis so other servers can route to this one
async function registerConnection(userId: string, connId: string): Promise<void> {
  await redis.hSet(`user:${userId}:connections`, connId, SERVER_ID);
  await redis.expire(`user:${userId}:connections`, 3600);
}

async function sendToUser(userId: string, payload: unknown): Promise<void> {
  const serverMap = await redis.hGetAll(`user:${userId}:connections`);
  const servers = new Set(Object.values(serverMap));

  for (const serverId of servers) {
    await redis.publish(`server:${serverId}`, JSON.stringify({ userId, payload }));
  }
}

This adds a Redis lookup per outbound message. The tradeoff is less fan-out waste at the cost of more reads. For most systems, the simple fan-out approach is fine until you’re in the thousands of messages per second range.

Consistent Hashing

When you need to route connections more deterministically, consistent hashing assigns users to server slots in a way that minimizes remapping when the fleet changes size.

import { createHash } from 'crypto';

class ConsistentHashRing {
  private ring = new Map<number, string>();
  private sortedKeys: number[] = [];
  private readonly replicas: number;

  constructor(replicas = 150) {
    this.replicas = replicas;
  }

  addNode(nodeId: string): void {
    for (let i = 0; i < this.replicas; i++) {
      const hash = this.hash(`${nodeId}:${i}`);
      this.ring.set(hash, nodeId);
    }
    this.sortedKeys = [...this.ring.keys()].sort((a, b) => a - b);
  }

  removeNode(nodeId: string): void {
    for (let i = 0; i < this.replicas; i++) {
      const hash = this.hash(`${nodeId}:${i}`);
      this.ring.delete(hash);
    }
    this.sortedKeys = [...this.ring.keys()].sort((a, b) => a - b);
  }

  getNode(key: string): string | null {
    if (this.ring.size === 0) return null;
    const hash = this.hash(key);
    const idx = this.sortedKeys.findIndex(k => k >= hash);
    const target = idx === -1 ? this.sortedKeys[0] : this.sortedKeys[idx];
    return this.ring.get(target) ?? null;
  }

  private hash(value: string): number {
    const buf = createHash('md5').update(value).digest();
    return buf.readUInt32BE(0);
  }
}

With consistent hashing, a user’s connection is directed to a predictable server. The connection itself still needs to be re-established if that server restarts, but the assignment is stable under scaling events.

Backpressure

A slow client is a resource leak. If you push messages faster than the client can consume them, they buffer in the socket’s send buffer. When the buffer fills, socket.send() will throw or silently drop messages depending on how you call it.

The ws library exposes socket.bufferedAmount (or you can inspect socket.readyState). A more robust approach uses the return value of socket.send() and the drain event:

function sendWithBackpressure(
  conn: Connection,
  data: string,
  onDropped?: () => void
): void {
  if (conn.socket.readyState !== WebSocket.OPEN) return;

  // bufferedAmount is the number of bytes queued but not yet sent
  const MAX_BUFFER = 1024 * 1024; // 1MB
  if ((conn.socket as any).bufferedAmount > MAX_BUFFER) {
    // Client is too far behind. Drop the message and log it.
    console.warn({ connId: conn.id }, 'dropping message: buffer full');
    onDropped?.();
    return;
  }

  conn.socket.send(data, (err) => {
    if (err) {
      console.error({ connId: conn.id, err }, 'send error');
    }
  });
}

For cases where you cannot drop messages (financial data, ordered event streams), you need a bounded queue per connection with a consumer loop that respects the socket’s drain state. But dropping is usually acceptable for real-time UI updates: the next state update makes the missed one irrelevant.

Reconnection Strategies

Clients should always attempt to reconnect on unexpected closure. The naive implementation opens a loop. Use exponential backoff with jitter:

class ReconnectingWebSocket {
  private socket: WebSocket | null = null;
  private attempt = 0;
  private readonly maxDelay = 30_000;
  private readonly baseDelay = 1_000;

  constructor(private readonly url: string) {
    this.connect();
  }

  private connect(): void {
    this.socket = new WebSocket(this.url);

    this.socket.addEventListener('open', () => {
      this.attempt = 0;
      console.log('connected');
    });

    this.socket.addEventListener('close', (event) => {
      // 1000 = clean close, don't reconnect
      if (event.code === 1000) return;

      const delay = this.backoff();
      console.log(`reconnecting in ${delay}ms (attempt ${this.attempt})`);
      setTimeout(() => this.connect(), delay);
    });

    this.socket.addEventListener('error', () => {
      // 'error' is always followed by 'close', handle reconnect there
    });
  }

  private backoff(): number {
    this.attempt++;
    const exp = Math.min(this.baseDelay * 2 ** this.attempt, this.maxDelay);
    // Add jitter: random value in [0, exp/2]
    return exp / 2 + Math.random() * (exp / 2);
  }

  send(data: string): void {
    if (this.socket?.readyState === WebSocket.OPEN) {
      this.socket.send(data);
    }
  }
}

The jitter prevents thundering herd: if your server restarts and 5,000 clients all wait exactly the same delay, they reconnect in a synchronized spike. Jitter spreads them across the window.

Tradeoffs: WebSocket vs SSE vs Long Polling

Dimension	WebSocket	SSE	Long Polling
Direction	Bidirectional	Server to client only	Server to client only
Protocol	Custom framing over TCP	HTTP/1.1 text stream	Standard HTTP
Proxy compatibility	Requires upgrade support	Generally compatible	Universal
Reconnection	Manual, client-side	Automatic (browser)	Automatic (next request)
Horizontal scaling	Complex (fan-out required)	Same complexity	Stateless, easy
Browser support	Universal	No IE11, limited in Edge Legacy	Universal
Message ordering	Guaranteed (TCP)	Guaranteed	Per-request ordering
Overhead per message	Low (framing is minimal)	Low (text stream)	High (full HTTP round-trip)
Idle connection cost	One TCP socket per client	One TCP socket per client	Minimal (polling interval)

SSE is underused. If your data flow is server-to-client (notifications, live dashboards, feed updates), SSE gives you HTTP semantics, automatic reconnection in the browser, and compatibility with standard CDNs and proxies. You lose bidirectionality, but many applications that reach for WebSockets don’t actually need it: the client sends data via regular POST requests and receives updates via SSE.

When Not to Use WebSockets

WebSockets are the right choice when you have frequent, low-latency bidirectional communication: collaborative editing, multiplayer games, live trading terminals. They are the wrong choice when:

The data flow is mostly one-directional. Notification systems, live dashboards, activity feeds. Use SSE. You get automatic reconnection, HTTP caching headers, standard proxy support, and less operational complexity.

Message frequency is low. If you’re delivering a few events per minute per client, the overhead of maintaining a persistent connection across your entire fleet is not justified. Long polling with a 30-second timeout performs comparably at low frequency and is trivially scalable.

You’re behind a restrictive proxy or corporate firewall. Many proxies do not support the WebSocket upgrade or will terminate idle connections aggressively. SSE or long polling over standard HTTPS port 443 passes through without issue.

Your load balancer doesn’t support WebSocket. Some managed environments and serverless platforms have connection time limits that make persistent WebSocket connections impractical. Cloudflare Workers with Durable Objects is one of the few serverless environments that handles WebSocket connections correctly; most others will close your connection at 30 seconds.

Production Considerations

Connection limits. Each WebSocket connection holds a file descriptor. Linux defaults to 1024 per process. Set ulimit -n and /etc/security/limits.conf before you hit this in production.

Memory per connection. Each connection holds socket buffers (default 4KB read, 4KB write in the kernel, plus application-level state). At 10,000 connections on a single server, plan for 200-400MB of memory just for connection state, before your application data.

Graceful shutdown. When your server needs to restart, send a close frame (1001 “going away”) to all clients before the process exits. This triggers client-side reconnection logic immediately rather than waiting for TCP timeout. Wire this into your SIGTERM handler.

process.on('SIGTERM', async () => {
  console.log('shutting down: closing connections');

  for (const conn of connectionManager.all()) {
    conn.socket.close(1001, 'server restarting');
  }

  // Wait for connections to drain before exiting
  await new Promise(resolve => setTimeout(resolve, 5_000));
  process.exit(0);
});

Observability. Track active connection count, message throughput (in and out), heartbeat timeout rate, and reconnection rate as metrics. A rising heartbeat timeout rate is your earliest signal that clients are experiencing network issues or your server is under memory pressure.

TLS termination. Terminate TLS at the load balancer, not at your application. The application speaks unencrypted WebSocket (ws://) to the load balancer, which handles wss:// toward the client. This keeps your CPU budget for application work rather than TLS handshakes.

The Pattern That Survives Scale

Start with a single server and the Redis pub/sub fan-out pattern, even before you need multiple instances. The fan-out wiring is the same code whether you have one server or twenty. When you add the second server, nothing changes.

Sticky sessions add complexity without solving the fundamental problem. They delay the day you have to implement fan-out, not eliminate it.

The connection lifecycle code, the heartbeat, and the backpressure handling are the same at 100 connections or 100,000. Get them right early. They are the parts that will actually cause your 3am pages.