Designing a Chat System: Real-Time Messaging, Presence, and Delivery Guarantees at Scale

Chat is a deceivingly hard problem. The happy path is trivial: two users online at the same time, message goes in, message comes out. But the edge cases are where real systems fail. What happens when the recipient is offline? What if both users send a message at the same moment? How do you guarantee a message was delivered exactly once at scale? How do you fan out a single message to 50,000 group members without melting your servers?

This article covers the full application layer of a production chat system: the message data model, ordering guarantees, delivery semantics, presence tracking, read receipts, and group fan-out strategies. WebSocket is the transport (covered in depth elsewhere), so here the focus is on what happens above the connection layer.

The Core Message Flow

Every chat system is fundamentally a pipeline: a client sends a message, the server stores it, and the server delivers it to recipients. The sequence matters.

// The basic message payload a client sends
interface SendMessageRequest {
  clientMessageId: string; // client-generated idempotency key
  conversationId: string;
  senderId: string;
  content: string;
  contentType: "text" | "image" | "file";
  clientTimestamp: number; // unix ms, for display only
}

// What the server stores
interface Message {
  messageId: string;        // server-generated, globally unique
  clientMessageId: string;  // for dedup, tied to senderId
  conversationId: string;
  senderId: string;
  content: string;
  contentType: "text" | "image" | "file";
  serverTimestamp: number;  // authoritative ordering timestamp
  sequenceNumber: number;   // monotonically increasing per conversation
  deliveryStatus: "stored" | "delivered" | "read";
}

The critical rule: store before deliver. If you deliver first and the storage write fails, you have a message the recipient received but that disappears on reload. Store the message to durable storage, then fan it out. The write to storage is the commit point.

The clientMessageId is how you handle retries. If a client sends a message, gets a network timeout, and retries, the server checks whether (senderId, clientMessageId) already exists. If it does, return the existing messageId without re-inserting. This is idempotent message submission.

Message Ordering with Logical Clocks

Client timestamps are unreliable. Clients have clock drift, and two clients sending messages at the “same time” will have different views of what time it is. If you order by client timestamp, you get non-deterministic ordering across users, which means different clients may show messages in different orders.

The solution is to use server-assigned sequence numbers per conversation.

class ConversationSequencer {
  private redis: Redis;

  async nextSequenceNumber(conversationId: string): Promise<number> {
    // Atomic increment in Redis
    const key = `seq:${conversationId}`;
    return await this.redis.incr(key);
  }

  async assignSequence(message: Omit<Message, "sequenceNumber">): Promise<Message> {
    const seq = await this.nextSequenceNumber(message.conversationId);
    return { ...message, sequenceNumber: seq };
  }
}

The sequence number becomes the authoritative order within a conversation. Clients always render messages sorted by sequenceNumber, not by clientTimestamp. The client timestamp stays on the record purely for display (showing “sent at 3:47 PM”).

For a single-instance server, a database auto-increment column works. For distributed systems, you need either a central sequencer (single point of failure, high throughput via Redis INCR) or a hybrid approach: use a coarse timestamp with a per-node sequence suffix, and resolve ties at read time. Twitter Snowflake IDs are a common pattern here: 41 bits of milliseconds, 10 bits of machine ID, 12 bits of sequence per millisecond.

// Snowflake-style ID generation
class SnowflakeIdGenerator {
  private epoch = 1700000000000; // custom epoch, Nov 2023
  private nodeId: number;        // 0-1023
  private sequence = 0;
  private lastTimestamp = -1;

  constructor(nodeId: number) {
    this.nodeId = nodeId & 0x3ff; // 10 bits
  }

  nextId(): bigint {
    let ts = Date.now() - this.epoch;

    if (ts === this.lastTimestamp) {
      this.sequence = (this.sequence + 1) & 0xfff; // 12 bits
      if (this.sequence === 0) {
        // Sequence overflow: wait for next millisecond
        while (ts <= this.lastTimestamp) {
          ts = Date.now() - this.epoch;
        }
      }
    } else {
      this.sequence = 0;
    }

    this.lastTimestamp = ts;

    return (
      (BigInt(ts) << 22n) |
      (BigInt(this.nodeId) << 12n) |
      BigInt(this.sequence)
    );
  }
}

These IDs sort lexicographically by time, give you total ordering without coordination across nodes, and are compact enough to use as database primary keys.

Delivery Guarantees: At-Least-Once with Deduplication

Most production chat systems implement at-least-once delivery, not exactly-once. Exactly-once across a network boundary is fundamentally impossible without coordination, and that coordination costs latency.

At-least-once means: the server will keep retrying delivery until the client acknowledges receipt. The client is responsible for deduplication on its end.

interface DeliveryAck {
  messageId: string;
  recipientId: string;
  ackedAt: number;
}

class MessageDeliveryManager {
  async deliverToOnlineClient(
    message: Message,
    recipientSocket: WebSocket
  ): Promise<void> {
    // Send and wait for ACK
    const ackPromise = this.waitForAck(message.messageId, recipientSocket);

    recipientSocket.send(JSON.stringify({
      type: "new_message",
      message,
    }));

    try {
      await Promise.race([
        ackPromise,
        this.timeout(5000), // 5s ACK timeout
      ]);
      await this.markDelivered(message.messageId, recipientSocket.userId);
    } catch {
      // ACK timeout: queue for retry
      await this.enqueueForRetry(message, recipientSocket.userId);
    }
  }

  async deduplicateIncoming(
    message: Message,
    recipientId: string
  ): Promise<boolean> {
    const key = `delivered:${recipientId}:${message.messageId}`;
    // SET NX with 24h TTL: returns true only if key was newly set
    const isNew = await this.redis.set(key, "1", "NX", "EX", 86400);
    return isNew !== null;
  }
}

The client tracks which messageIds it has already rendered. On reconnect, it requests messages since its last known sequence number:

// Client reconnect sync
async function syncOnReconnect(
  conversationId: string,
  lastKnownSeq: number
): Promise<Message[]> {
  const response = await fetch(
    `/api/conversations/${conversationId}/messages?after_seq=${lastKnownSeq}`
  );
  return response.json();
}

This handles the common case: a client goes offline, misses some messages, reconnects, and catches up. The server just queries the messages table by (conversationId, sequenceNumber > lastKnownSeq).

Presence Tracking

Presence (online/offline/last seen) looks simple but has significant design pressure at scale. You can’t just set a flag in a database on connect and disconnect, because connections drop silently all the time. You need a heartbeat-based model.

class PresenceService {
  private redis: Redis;
  private PRESENCE_TTL = 65; // seconds, slightly more than heartbeat interval

  async heartbeat(userId: string, connectionId: string): Promise<void> {
    const key = `presence:${userId}`;
    const now = Date.now();

    await this.redis.pipeline()
      .hset(key, connectionId, now.toString())
      .expire(key, this.PRESENCE_TTL)
      .exec();
  }

  async onDisconnect(userId: string, connectionId: string): Promise<void> {
    const key = `presence:${userId}`;
    await this.redis.hdel(key, connectionId);

    // Check if any connections remain
    const remaining = await this.redis.hlen(key);
    if (remaining === 0) {
      await this.publishOffline(userId);
    }
  }

  async isOnline(userId: string): Promise<boolean> {
    const key = `presence:${userId}`;
    const connections = await this.redis.hgetall(key);
    if (!connections) return false;

    const now = Date.now();
    const STALE_THRESHOLD = 60_000; // 60s

    // A user is online if any connection has a recent heartbeat
    return Object.values(connections).some(
      (ts) => now - parseInt(ts) < STALE_THRESHOLD
    );
  }
}

Clients send a heartbeat every 30 seconds. The Redis key expires after 65 seconds. If the client disconnects cleanly, the server deletes the connection entry immediately. If the connection dies silently (process kill, network drop), the key just expires. Either way, presence state self-heals.

For 1:1 chats, presence events can be pushed directly to the conversation partner via their WebSocket connection. For group chats with thousands of members, you don’t broadcast presence changes to everyone; you deliver presence on-demand when a user opens a conversation.

Read Receipts and Typing Indicators

Read receipts are a cursor per user per conversation, tracking the highest sequence number they have seen.

interface ReadCursor {
  userId: string;
  conversationId: string;
  lastReadSeq: number;
  updatedAt: number;
}

class ReadReceiptService {
  async markRead(
    userId: string,
    conversationId: string,
    upToSeq: number
  ): Promise<void> {
    // Upsert: only advance the cursor, never go backward
    await this.db.query(`
      INSERT INTO read_cursors (user_id, conversation_id, last_read_seq, updated_at)
      VALUES ($1, $2, $3, $4)
      ON CONFLICT (user_id, conversation_id)
      DO UPDATE SET
        last_read_seq = GREATEST(read_cursors.last_read_seq, EXCLUDED.last_read_seq),
        updated_at = EXCLUDED.updated_at
    `, [userId, conversationId, upToSeq, Date.now()]);

    // Push the receipt to the conversation partner(s)
    await this.fanOutReadReceipt(userId, conversationId, upToSeq);
  }

  async getUnreadCount(
    userId: string,
    conversationId: string
  ): Promise<number> {
    const result = await this.db.query(`
      SELECT COUNT(*) FROM messages m
      LEFT JOIN read_cursors rc ON
        rc.user_id = $1 AND rc.conversation_id = m.conversation_id
      WHERE m.conversation_id = $2
        AND m.sender_id != $1
        AND m.sequence_number > COALESCE(rc.last_read_seq, 0)
    `, [userId, conversationId]);

    return parseInt(result.rows[0].count);
  }
}

The GREATEST in the upsert is critical. If two ACKs arrive out of order (network reordering), you never regress the cursor.

Typing indicators are ephemeral and should never touch a database. They go through the pub/sub layer directly, with a short TTL so they self-expire if the typing event is missed.

class TypingIndicatorService {
  async startTyping(userId: string, conversationId: string): Promise<void> {
    const key = `typing:${conversationId}:${userId}`;
    // 5 second TTL: if no update, typing stops automatically
    await this.redis.setex(key, 5, "1");
    await this.pubsub.publish(`conv:${conversationId}`, {
      type: "typing_start",
      userId,
      conversationId,
    });
  }

  async getTypingUsers(conversationId: string): Promise<string[]> {
    const pattern = `typing:${conversationId}:*`;
    const keys = await this.redis.keys(pattern);
    return keys.map((k) => k.split(":")[2]);
  }
}

Clients send a typing_start event on keypress (debounced, at most once every 3 seconds) and a typing_stop on send or focus loss. The TTL handles the case where a user closes the app mid-message without sending a stop event.

Group Chat Fan-Out

This is where chat systems break at scale. A message to a 1:1 conversation needs to reach one person. A message to a group with 500 members needs to reach 500 connections, potentially spread across dozens of servers.

Two strategies dominate:

Push fan-out (write-time): When a message is sent, immediately enqueue a delivery task for every group member. Low read latency, high write amplification.

Pull fan-out (read-time): Store the message once. When a client connects or opens a conversation, it pulls missed messages. High read latency for large groups (everyone queries on reconnect), but write amplification is minimal.

class GroupMessageFanOut {
  // Push fan-out: good for small-to-medium groups
  async fanOutSmallGroup(
    message: Message,
    memberIds: string[]
  ): Promise<void> {
    // Batch into chunks to avoid overwhelming the queue
    const CHUNK_SIZE = 100;

    for (let i = 0; i < memberIds.length; i += CHUNK_SIZE) {
      const chunk = memberIds.slice(i, i + CHUNK_SIZE);
      await this.messageQueue.publishBatch(
        chunk.map((recipientId) => ({
          type: "deliver_message",
          message,
          recipientId,
        }))
      );
    }
  }

  // Hybrid fan-out: push to online users, let offline users pull
  async fanOutHybrid(
    message: Message,
    memberIds: string[]
  ): Promise<void> {
    const onlineMembers = await this.presence.filterOnline(memberIds);
    const offlineMembers = memberIds.filter(
      (id) => !onlineMembers.includes(id)
    );

    // Push to online members immediately
    if (onlineMembers.length > 0) {
      await this.fanOutSmallGroup(message, onlineMembers);
    }

    // Offline members will pull on reconnect, no action needed
    // (messages are already stored, they sync by sequenceNumber)
    void offlineMembers; // no-op: handled by reconnect sync
  }
}

For very large groups (tens of thousands), even online-only push fan-out is expensive. The pattern used in production is to segment members into buckets and use a scatter-gather approach: a router dispatches to per-region or per-shard delivery workers, each responsible for a subset of members.

The threshold matters. Groups under 500 members can reasonably use push fan-out. Groups above that threshold should consider hybrid or pull-based delivery, with an in-app notification count pushed to indicate new activity without delivering the full message content.

Offline Message Queuing

When a recipient is offline, messages must be queued and delivered on reconnect. The simplest correct approach: since messages are stored in order with sequence numbers, reconnect sync (described above) naturally handles offline delivery. The client connects, sends its last known sequenceNumber, and the server returns everything it missed.

The complication is push notifications. When a user is offline, you want to send a push notification (FCM, APNs) to wake their app. This requires a separate delivery path.

class OfflineNotificationService {
  async notifyOfflineUser(
    recipientId: string,
    message: Message
  ): Promise<void> {
    const deviceTokens = await this.getDeviceTokens(recipientId);

    if (deviceTokens.length === 0) return;

    const sender = await this.getUserDisplayName(message.senderId);
    const preview = this.buildPreview(message);

    await this.pushProvider.sendBatch(
      deviceTokens.map((token) => ({
        token,
        notification: {
          title: sender,
          body: preview,
        },
        data: {
          conversationId: message.conversationId,
          messageId: message.messageId,
          // Deep link into the conversation
        },
      }))
    );
  }

  private buildPreview(message: Message): string {
    if (message.contentType === "text") {
      return message.content.slice(0, 100);
    }
    if (message.contentType === "image") {
      return "Sent an image";
    }
    return "Sent a file";
  }
}

The notification is fire-and-forget. The actual message content arrives via the pull sync when the app opens. Never rely on push notification delivery as proof the message was received.

Push vs Pull: Tradeoffs

Dimension	Push (Server-Initiated)	Pull (Client-Initiated)
Latency	Sub-second for online users	Depends on poll interval
Server load	High write amplification for groups	High read load on reconnect
Offline handling	Requires separate queue	Natural (pull after reconnect)
Simplicity	Complex fan-out logic	Simple: query by sequence
Missed messages	Risk if queue drops event	None: storage is authoritative
Ordering	Must sequence events in queue	Trivially sorted by seq number

Most production systems use a hybrid: push for online users (low latency), pull for reconnect sync (reliability). The push path is best-effort; the pull path is the source of truth.

Production Considerations

Storage schema. Partition the messages table by conversation_id to keep hot conversations on the same shard. Use a composite index on (conversation_id, sequence_number) for range scans. For older messages, move to cold storage (object storage or a cheaper tier) after 90 days. Most users never scroll back more than a few hundred messages.

Connection routing. Each WebSocket connection lives on a specific server. When server A needs to deliver a message to a user connected to server B, it publishes to a pub/sub channel (Redis Pub/Sub or a message broker) that server B subscribes to. Each server subscribes to channels for its own connections. This is the standard horizontal scaling pattern for WebSocket servers.

class ConnectionRouter {
  // When a user connects, register their server affinity
  async registerConnection(
    userId: string,
    connectionId: string,
    serverId: string
  ): Promise<void> {
    const key = `conn:${userId}`;
    await this.redis.hset(key, connectionId, serverId);
    await this.redis.expire(key, 3600);

    // Subscribe to this user's delivery channel
    await this.pubsub.subscribe(`user:${userId}`, (message) => {
      this.localDelivery.deliver(connectionId, message);
    });
  }

  async routeMessage(recipientId: string, message: Message): Promise<void> {
    // Publish to the user's channel; whichever server holds the
    // connection will pick it up
    await this.pubsub.publish(`user:${recipientId}`, {
      type: "new_message",
      message,
    });
  }
}

Rate limiting. Chat is vulnerable to abuse. Rate-limit message sends per user per conversation (e.g., 10 messages per second) and per user globally (e.g., 100 messages per minute). Enforce at the API gateway layer before the message hits storage.

Message size limits. Cap individual message content at 64KB for text. For images and files, accept a reference (object storage URL) rather than the binary payload. Clients upload files directly to object storage and send the URL as the message content.

Idempotency window. The (senderId, clientMessageId) dedup check needs a TTL. Keeping it forever is storage-expensive and unnecessary. A 24-hour window is sufficient; if a client retries a message 24 hours later, it’s a logic error, not a network retry.

Conversation metadata. Store a conversations table with last_message_at, last_message_preview, and per-user unread counts as a denormalized cache. Rebuilding these on every inbox load from raw messages is too expensive. Update them as a side effect of message storage.

The Hardest Part

Message ordering in distributed systems is the part that trips up most implementations. The temptation is to trust client timestamps. Don’t. Two phones on different continents, both sending a message to the same group at the “same time,” will each see the other’s message arriving “after” their own based on their local clock. A monotonically increasing server-assigned sequence number per conversation removes the ambiguity entirely.

The second trap is treating delivery as a guarantee. WebSocket connections drop. Mobile devices go into low-power mode. The server push is a best-effort optimization. The pull sync on reconnect is the guarantee. Build the pull path first, verify it works correctly, then layer push delivery on top as an optimization. The system degrades gracefully to polling if the push path fails.

A well-designed chat system isn’t complicated. It’s a durable message store with a sequence-number index, a fan-out layer that respects online/offline state, and a simple cursor per user per conversation tracking what they’ve seen. Everything else (read receipts, typing indicators, presence) is just state management layered on top of that foundation.