System Design ·

Designing a Live Commerce Platform: Real-Time Bidding, Inventory Reservation, and Multi-Channel Product Sync at Scale

Live commerce platforms combine the worst concurrency problems from flash sales, real-time auctions, and multi-channel inventory sync into a single user session. This guide covers the backend architecture: WebSocket fan-out for bidding, pessimistic reservation under concurrent viewer load, multi-channel catalog sync across TikTok Shop and Instagram, and edge-rendered shoppable video.

Designing a Live Commerce Platform: Real-Time Bidding, Inventory Reservation, and Multi-Channel Product Sync at Scale

Live commerce is the most unforgiving intersection of eCommerce and real-time systems. You have thousands of concurrent viewers watching a stream, an auctioneer dropping prices every few seconds, and a catalog simultaneously synced across TikTok Shop, Instagram, and YouTube Shopping. A single race condition causes overselling. A slow bid update costs the platform credibility. A catalog desync shows a buyer a product that sold out on a different channel three minutes ago.

The engineering problems are not novel in isolation. Fan-out, inventory locking, and API synchronization are well-understood patterns. What makes live commerce hard is that they all converge in the same user session, under extreme read-write imbalance, with a time-boxed urgency that punishes latency in ways a standard checkout flow never does.

This guide covers the backend architecture of a live commerce platform: the bidding engine, the inventory reservation system, multi-channel catalog sync, and the edge rendering layer for shoppable video. Each section includes concrete TypeScript and tradeoff discussion.

The Bidding Engine: WebSocket Fan-Out Under Load

A live auction emits state updates at high frequency: price drops, bid placements, countdown ticks, and winner announcements. Every connected viewer needs to see the same state within a few hundred milliseconds.

The naive approach sends updates directly to every connected client from the bid processing service. This works at 50 viewers. It falls apart at 5,000.

The production pattern separates bid processing from fan-out. Bids are processed by a stateless service that writes to a Redis Stream. A fan-out tier subscribes to that stream and pushes updates over WebSocket connections held in memory. This means your bid processing service never holds connection state, and your fan-out layer can scale horizontally.

// bid-processor.ts
import { Redis } from "ioredis";
import { z } from "zod";

const BidSchema = z.object({
  auctionId: z.string().uuid(),
  bidderId: z.string().uuid(),
  amount: z.number().positive(),
  timestamp: z.number(),
});

type Bid = z.infer<typeof BidSchema>;

interface AuctionState {
  currentPrice: number;
  leadingBidder: string | null;
  endsAt: number;
  status: "active" | "closing" | "sold" | "passed";
}

export class BidProcessor {
  constructor(
    private readonly redis: Redis,
    private readonly streamKey: string = "auction:events"
  ) {}

  async placeBid(rawBid: unknown): Promise<{ accepted: boolean; reason?: string }> {
    const parsed = BidSchema.safeParse(rawBid);
    if (!parsed.success) {
      return { accepted: false, reason: "invalid_bid" };
    }

    const bid = parsed.data;
    const stateKey = `auction:state:${bid.auctionId}`;

    // Use a Lua script for atomic read-compare-write
    const script = `
      local state = redis.call('HGETALL', KEYS[1])
      if #state == 0 then return {err='auction_not_found'} end
      local current = {}
      for i = 1, #state, 2 do
        current[state[i]] = state[i+1]
      end
      if current['status'] ~= 'active' then return {err='auction_not_active'} end
      if tonumber(ARGV[1]) <= tonumber(current['currentPrice']) then
        return {err='bid_too_low'}
      end
      redis.call('HSET', KEYS[1], 'currentPrice', ARGV[1], 'leadingBidder', ARGV[2])
      return 'ok'
    `;

    const result = await this.redis.eval(
      script,
      1,
      stateKey,
      bid.amount.toString(),
      bid.bidderId
    );

    if (result !== "ok") {
      return { accepted: false, reason: result as string };
    }

    // Publish to stream for fan-out
    await this.redis.xadd(
      this.streamKey,
      "*",
      "type", "bid_accepted",
      "auctionId", bid.auctionId,
      "amount", bid.amount.toString(),
      "bidderId", bid.bidderId,
      "ts", bid.timestamp.toString()
    );

    return { accepted: true };
  }
}

The fan-out service subscribes to the Redis Stream using XREAD BLOCK and routes events to the correct WebSocket group. Each auction maps to a room. Viewers join the room on stream load.

// fanout-service.ts
import { WebSocketServer, WebSocket } from "ws";
import { Redis } from "ioredis";

type Room = Set<WebSocket>;

export class FanoutService {
  private rooms = new Map<string, Room>();
  private lastId = "0-0";

  constructor(
    private readonly redis: Redis,
    private readonly wss: WebSocketServer,
    private readonly streamKey: string = "auction:events"
  ) {
    this.wss.on("connection", (ws, req) => {
      const auctionId = this.extractAuctionId(req.url ?? "");
      if (!auctionId) return ws.close();
      this.joinRoom(auctionId, ws);
      ws.on("close", () => this.leaveRoom(auctionId, ws));
    });
  }

  async startConsuming(): Promise<void> {
    while (true) {
      const results = await this.redis.xread(
        "BLOCK", 100,
        "COUNT", 100,
        "STREAMS", this.streamKey, this.lastId
      ) as Array<[string, Array<[string, string[]]>]> | null;

      if (!results) continue;

      for (const [, messages] of results) {
        for (const [id, fields] of messages) {
          this.lastId = id;
          const event = this.parseFields(fields);
          this.broadcast(event.auctionId, JSON.stringify(event));
        }
      }
    }
  }

  private broadcast(auctionId: string, payload: string): void {
    const room = this.rooms.get(auctionId);
    if (!room) return;
    for (const ws of room) {
      if (ws.readyState === WebSocket.OPEN) {
        ws.send(payload);
      }
    }
  }

  private joinRoom(auctionId: string, ws: WebSocket): void {
    if (!this.rooms.has(auctionId)) {
      this.rooms.set(auctionId, new Set());
    }
    this.rooms.get(auctionId)!.add(ws);
  }

  private leaveRoom(auctionId: string, ws: WebSocket): void {
    this.rooms.get(auctionId)?.delete(ws);
  }

  private extractAuctionId(url: string): string | null {
    const match = url.match(/\/auctions\/([a-f0-9-]+)/);
    return match?.[1] ?? null;
  }

  private parseFields(fields: string[]): Record<string, string> {
    const obj: Record<string, string> = {};
    for (let i = 0; i < fields.length; i += 2) {
      obj[fields[i]] = fields[i + 1];
    }
    return obj;
  }
}

At scale, a single fan-out node cannot hold all WebSocket connections. You route viewers to fan-out nodes by auction ID using consistent hashing at the load balancer. All fan-out nodes subscribe to the same Redis Stream but filter by auction ID, or you use a Redis Pub/Sub channel per auction and have the fan-out node subscribe only to active auction channels on its node.

Inventory Reservation: Preventing Overselling Under Concurrent Load

Flash sales are a solved problem in theory and a recurring disaster in practice. The core issue: your read and write are separated in time. You read available inventory, decide to allow a purchase, and write the decrement. Between the read and the write, another session can read the same count and make the same decision. Two buyers claim the last unit.

The fix is to make the read and the decrement atomic. Redis DECRBY with a guard is the standard approach for high-throughput cases. For financial-grade correctness, you use a Postgres SELECT FOR UPDATE with a row-level lock.

The choice between Redis-first and Postgres-first reservation depends on your traffic and consistency requirements.

// inventory-service.ts
import { Redis } from "ioredis";
import { Pool } from "pg";

export class InventoryService {
  constructor(
    private readonly redis: Redis,
    private readonly pg: Pool
  ) {}

  // Redis-first: fast, suitable for high-burst scenarios
  // Risk: Redis reservation can diverge from DB if crash occurs
  async reserveWithRedis(
    productId: string,
    quantity: number,
    reservationId: string,
    ttlSeconds: number = 300
  ): Promise<{ reserved: boolean; remaining?: number }> {
    const inventoryKey = `inventory:${productId}`;
    const reservationKey = `reservation:${reservationId}`;

    const script = `
      local current = tonumber(redis.call('GET', KEYS[1]))
      if current == nil then return {err='no_inventory_record'} end
      if current < tonumber(ARGV[1]) then return {err='insufficient_stock'} end
      redis.call('DECRBY', KEYS[1], ARGV[1])
      redis.call('SET', KEYS[2], ARGV[1], 'EX', ARGV[2])
      return redis.call('GET', KEYS[1])
    `;

    const result = await this.redis.eval(
      script,
      2,
      inventoryKey,
      reservationKey,
      quantity.toString(),
      ttlSeconds.toString()
    );

    if (typeof result === "object" && "err" in (result as object)) {
      return { reserved: false };
    }

    return { reserved: true, remaining: Number(result) };
  }

  // Postgres-first: strong consistency, lower throughput ceiling
  // Use for lower-volume, high-value items (luxury goods, limited editions)
  async reserveWithPostgres(
    productId: string,
    quantity: number,
    reservationId: string
  ): Promise<{ reserved: boolean; remaining?: number }> {
    const client = await this.pg.connect();

    try {
      await client.query("BEGIN");

      const { rows } = await client.query<{ available: number }>(
        `SELECT available FROM inventory WHERE product_id = $1 FOR UPDATE`,
        [productId]
      );

      if (!rows[0] || rows[0].available < quantity) {
        await client.query("ROLLBACK");
        return { reserved: false };
      }

      await client.query(
        `UPDATE inventory SET available = available - $1 WHERE product_id = $2`,
        [quantity, productId]
      );

      await client.query(
        `INSERT INTO reservations (id, product_id, quantity, expires_at)
         VALUES ($1, $2, $3, NOW() + INTERVAL '5 minutes')`,
        [reservationId, productId, quantity]
      );

      await client.query("COMMIT");
      return { reserved: true, remaining: rows[0].available - quantity };
    } catch (err) {
      await client.query("ROLLBACK");
      throw err;
    } finally {
      client.release();
    }
  }

  async releaseReservation(reservationId: string, productId: string): Promise<void> {
    const client = await this.pg.connect();
    try {
      await client.query("BEGIN");
      const { rows } = await client.query<{ quantity: number }>(
        `DELETE FROM reservations WHERE id = $1 RETURNING quantity`,
        [reservationId]
      );
      if (rows[0]) {
        await client.query(
          `UPDATE inventory SET available = available + $1 WHERE product_id = $2`,
          [rows[0].quantity, productId]
        );
        // Return the unit to Redis cache as well
        await this.redis.incrby(`inventory:${productId}`, rows[0].quantity);
      }
      await client.query("COMMIT");
    } finally {
      client.release();
    }
  }
}

For a live commerce platform with high viewer counts, Redis-first reservation with a short TTL is the practical choice for the reservation window. A background job reconciles Redis state against Postgres every 30 seconds and releases expired reservations. The Postgres row is the source of truth for fulfillment; Redis is the traffic-shaping layer.

Multi-Channel Product Catalog Sync

TikTok Shop, Instagram Shopping, and YouTube Shopping each have distinct catalog schemas, authentication patterns, and rate limits. The fundamental problem is that your canonical product catalog (price, title, images, inventory status) must stay consistent across three external systems that have no native coordination mechanism.

The architecture uses a Catalog Event Bus. Every mutation to the canonical catalog (price update, inventory status change, new product publish) emits a typed event. A set of channel-specific adapter services consumes those events and translates them to the target API’s schema and rate constraints.

// catalog-sync.ts
import { z } from "zod";

const CatalogEventSchema = z.discriminatedUnion("type", [
  z.object({
    type: z.literal("product.updated"),
    productId: z.string(),
    fields: z.object({
      title: z.string().optional(),
      price: z.number().optional(),
      inventoryStatus: z.enum(["in_stock", "out_of_stock", "limited"]).optional(),
      imageUrls: z.array(z.string()).optional(),
    }),
    updatedAt: z.string().datetime(),
  }),
  z.object({
    type: z.literal("product.delisted"),
    productId: z.string(),
    updatedAt: z.string().datetime(),
  }),
]);

type CatalogEvent = z.infer<typeof CatalogEventSchema>;

interface ChannelAdapter {
  channel: string;
  push(event: CatalogEvent): Promise<{ success: boolean; error?: string }>;
}

export class TikTokShopAdapter implements ChannelAdapter {
  readonly channel = "tiktok_shop";

  constructor(
    private readonly baseUrl: string,
    private readonly appKey: string,
    private readonly accessToken: string
  ) {}

  async push(event: CatalogEvent): Promise<{ success: boolean; error?: string }> {
    if (event.type === "product.delisted") {
      return this.delistProduct(event.productId);
    }

    const payload = this.mapToTikTokSchema(event);
    const res = await fetch(`${this.baseUrl}/product/202309/products/${event.productId}`, {
      method: "PUT",
      headers: {
        "Content-Type": "application/json",
        "x-tts-access-token": this.accessToken,
      },
      body: JSON.stringify(payload),
    });

    if (!res.ok) {
      const body = await res.text();
      return { success: false, error: `TikTok API error ${res.status}: ${body}` };
    }

    return { success: true };
  }

  private mapToTikTokSchema(event: Extract<CatalogEvent, { type: "product.updated" }>) {
    return {
      title: event.fields.title,
      price: event.fields.price
        ? { amount: (event.fields.price * 100).toString(), currency: "USD" }
        : undefined,
      is_not_for_sale: event.fields.inventoryStatus === "out_of_stock",
    };
  }

  private async delistProduct(productId: string): Promise<{ success: boolean; error?: string }> {
    const res = await fetch(`${this.baseUrl}/product/202309/products/deactivate`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "x-tts-access-token": this.accessToken,
      },
      body: JSON.stringify({ product_ids: [productId] }),
    });
    return { success: res.ok };
  }
}

export class CatalogSyncOrchestrator {
  constructor(private readonly adapters: ChannelAdapter[]) {}

  async sync(event: CatalogEvent): Promise<void> {
    const results = await Promise.allSettled(
      this.adapters.map((adapter) => adapter.push(event))
    );

    for (let i = 0; i < results.length; i++) {
      const result = results[i];
      const adapter = this.adapters[i];
      if (result.status === "rejected" || !result.value.success) {
        const reason = result.status === "rejected"
          ? result.reason
          : result.value.error;
        // Emit to dead-letter queue for retry with exponential backoff
        console.error(`Sync failed for channel ${adapter.channel}:`, reason);
      }
    }
  }
}

Rate limits are the real operational challenge. TikTok Shop’s catalog API allows roughly 100 requests per minute per app. If you have 5,000 products and push a price update during a live event, you need a queue with rate-limiting per channel, not a direct fan-out. Use a channel-specific queue (BullMQ, SQS, or a Postgres-backed outbox) with token bucket enforcement per adapter.

Tradeoffs: Eventual Consistency vs. Strong Consistency for Inventory

ScenarioApproachConsistencyThroughputRisk
Flash sale (mass-market, < $50)Redis atomic decrement + async Postgres syncEventualVery highTemporary oversell if Redis crashes before sync
Bidding item (unique or limited)Postgres SELECT FOR UPDATE with read-committedStrongMediumLatency under lock contention at peak
Multi-channel catalog statusEvent bus with per-channel retry queueEventualHighChannel desync window up to 60s
Order confirmationPostgres transaction with reservation finalizationStrongLow-mediumRow lock on high-demand SKU
Reservation TTL expiryBackground job every 30s + Redis TTL triggerEventualN/AInventory held slightly beyond intended window

The practical position for a live commerce platform is: use eventual consistency at the display layer (showing inventory counts to viewers) and strong consistency at the purchase boundary (the moment a buyer submits an order). This matches the actual business risk. Showing a viewer “3 remaining” when it is actually 2 is cosmetically acceptable. Charging a buyer for a unit you cannot fulfill is not.

Low-Latency Edge Rendering for Shoppable Video

Shoppable video overlays product tiles on live or recorded streams. The latency requirement is tight: a viewer taps a product and expects the tile to appear with price and availability in under 200ms globally.

The rendering strategy is: pre-render product tiles at the CDN edge, hydrated with catalog data, and revalidate on every catalog event. Cloudflare Workers with KV or R2 is a practical fit. The product tile is a small JSON blob (title, price, image, availability status) cached at the edge. On a catalog event, you purge or revalidate the relevant edge keys.

// edge-product-tile.ts (Cloudflare Worker)
interface ProductTile {
  productId: string;
  title: string;
  price: number;
  inventoryStatus: "in_stock" | "out_of_stock" | "limited";
  imageUrl: string;
  cacheAge: number;
}

export async function handleRequest(
  request: Request,
  env: { PRODUCT_KV: KVNamespace; CATALOG_API: string; PURGE_SECRET: string }
): Promise<Response> {
  const url = new URL(request.url);

  // Purge endpoint called by catalog sync on product update
  if (url.pathname.startsWith("/purge/") && request.method === "POST") {
    const secret = request.headers.get("x-purge-secret");
    if (secret !== env.PURGE_SECRET) {
      return new Response("Unauthorized", { status: 401 });
    }
    const productId = url.pathname.split("/purge/")[1];
    await env.PRODUCT_KV.delete(`tile:${productId}`);
    return new Response("Purged", { status: 200 });
  }

  // Product tile fetch — serves viewer overlay requests
  const productId = url.pathname.split("/tiles/")[1];
  if (!productId) return new Response("Not Found", { status: 404 });

  const cached = await env.PRODUCT_KV.get<ProductTile>(`tile:${productId}`, "json");
  if (cached) {
    return new Response(JSON.stringify(cached), {
      headers: {
        "Content-Type": "application/json",
        "Cache-Control": "public, max-age=30",
        "X-Cache": "HIT",
      },
    });
  }

  // Cache miss: fetch from origin and populate KV
  const origin = await fetch(`${env.CATALOG_API}/products/${productId}/tile`);
  if (!origin.ok) return new Response("Not Found", { status: 404 });

  const tile = await origin.json<ProductTile>();
  tile.cacheAge = Date.now();

  await env.PRODUCT_KV.put(`tile:${productId}`, JSON.stringify(tile), {
    expirationTtl: 300, // 5 minutes — catalog sync purges earlier on change
  });

  return new Response(JSON.stringify(tile), {
    headers: {
      "Content-Type": "application/json",
      "Cache-Control": "public, max-age=30",
      "X-Cache": "MISS",
    },
  });
}

The catalog sync adapter calls the purge endpoint on each successful channel push. This keeps edge-cached tiles consistent with catalog changes within seconds rather than waiting for TTL expiry.

The Order Flow: Event-Driven and Idempotent

A live commerce order involves: inventory reservation, payment authorization, order record creation, and post-purchase events (confirmation email, fulfillment system notification, social proof signal to stream). Each step can fail independently.

Model this as an event-driven saga with idempotency keys at each stage.

// order-flow.ts
import { z } from "zod";

const OrderCommandSchema = z.object({
  idempotencyKey: z.string().uuid(),
  buyerId: z.string().uuid(),
  productId: z.string().uuid(),
  quantity: z.number().int().positive(),
  reservationId: z.string().uuid(),
  paymentMethodId: z.string(),
  channelSource: z.enum(["tiktok_shop", "instagram", "youtube", "direct"]),
});

type OrderCommand = z.infer<typeof OrderCommandSchema>;

type OrderStep =
  | { step: "inventory_confirmed"; reservationId: string }
  | { step: "payment_authorized"; chargeId: string }
  | { step: "order_created"; orderId: string }
  | { step: "fulfillment_notified"; fulfillmentId: string }
  | { step: "failed"; at: string; reason: string };

interface OrderOrchestrator {
  process(command: OrderCommand): AsyncGenerator<OrderStep>;
}

export class LiveCommerceOrderOrchestrator implements OrderOrchestrator {
  constructor(
    private readonly inventory: { confirm(reservationId: string): Promise<boolean> },
    private readonly payments: { authorize(methodId: string, amount: number, key: string): Promise<string> },
    private readonly orders: { create(data: object): Promise<string> },
    private readonly fulfillment: { notify(orderId: string): Promise<string> },
    private readonly eventBus: { emit(event: string, data: object): Promise<void> }
  ) {}

  async *process(command: OrderCommand): AsyncGenerator<OrderStep> {
    // Step 1: Confirm inventory reservation
    const confirmed = await this.inventory.confirm(command.reservationId);
    if (!confirmed) {
      yield { step: "failed", at: "inventory_confirmation", reason: "reservation_expired" };
      return;
    }
    yield { step: "inventory_confirmed", reservationId: command.reservationId };

    // Step 2: Authorize payment
    let chargeId: string;
    try {
      chargeId = await this.payments.authorize(
        command.paymentMethodId,
        0, // price resolved from reservation
        `${command.idempotencyKey}:payment`
      );
    } catch (err) {
      yield { step: "failed", at: "payment_authorization", reason: String(err) };
      return;
    }
    yield { step: "payment_authorized", chargeId };

    // Step 3: Create order record
    const orderId = await this.orders.create({
      buyerId: command.buyerId,
      productId: command.productId,
      quantity: command.quantity,
      chargeId,
      channelSource: command.channelSource,
    });
    yield { step: "order_created", orderId };

    // Step 4: Notify fulfillment (non-blocking for order response)
    const fulfillmentId = await this.fulfillment.notify(orderId);
    yield { step: "fulfillment_notified", fulfillmentId };

    // Post-purchase events are fire-and-forget
    void this.eventBus.emit("order.completed", {
      orderId,
      buyerId: command.buyerId,
      productId: command.productId,
      channelSource: command.channelSource,
    });
  }
}

Each step uses the idempotency key as a prefix so retries are safe. If the payment step fails after the inventory step succeeds, a compensating transaction releases the reservation. The generator pattern makes the saga steps explicit and testable in isolation.

Production Considerations

WebSocket connection limits. A single node process handles roughly 10K-60K WebSocket connections depending on message frequency. For 100K concurrent viewers across 10 simultaneous auctions, you need horizontal fan-out nodes with auction-affinity routing. Monitor connections-per-node as a primary capacity metric, not just CPU.

Redis stream backpressure. If your fan-out consumers fall behind the bid stream, entries accumulate. Set a MAXLEN on the stream with the ~ approximate trimming option to bound memory use. Consumers should checkpoint their lastId to recover from restarts without reprocessing the full history.

Reservation expiry under load. TTL-based expiry via Redis is non-deterministic at high load. A buyer who reaches checkout 4m 58s into a 5-minute reservation window may find their reservation expired before they complete the form. Add a 30-second grace period to the displayed countdown and extend the reservation TTL by 60 seconds on cart-page load.

Multi-channel rate limit budgeting. TikTok Shop, Meta Commerce API, and YouTube Shopping each have distinct rate limit tiers. During a live event, a single hot product can generate hundreds of inventory status updates per minute. Use a rate-limit-aware queue per channel with token bucket enforcement. Budget your API quota across background catalog sync and live-event push separately.

Inventory cache warming. Before a live event starts, warm the Redis inventory cache for all SKUs featured in the event. A cold cache during the first bid avalanche will spike Postgres load. Pre-event warm-up is a standard pre-flight step.

Cross-channel oversell detection. Even with Redis reservation, a product sold on TikTok Shop via their native checkout (bypassing your platform) can deplete physical inventory without triggering your reservation system. Poll channel order APIs every 60 seconds during live events and update your Redis inventory accordingly. Most channel APIs offer a webhook for this; implement the webhook as your primary path and the poll as a reconciliation backstop.

The Underlying Pattern

Live commerce compresses problems that traditional eCommerce handles sequentially into a time window measured in seconds. The architecture answers this by separating the hot path (bids, reservations) from the cold path (catalog sync, fulfillment), using the right consistency model for each boundary, and pushing read paths to the edge.

The bidding engine is fast because it delegates truth to Redis and delegates broadcast to a separate fan-out layer. The inventory system is correct because it treats the purchase confirmation as a strong consistency boundary while tolerating eventual consistency at the display layer. The catalog sync is durable because it uses a retry queue rather than direct API calls. The edge rendering is fast because product tiles are pre-rendered and invalidated on change rather than computed per request.

Each of these is a narrow, composable decision. The platform as a whole works because the boundaries between them are explicit and the consistency model at each boundary is chosen deliberately rather than inherited by default.

More in System Design

How Amazon Aurora Works Internally: The Log-Is-the-Database Architecture, Quorum Writes, and Storage-Compute Separation Behind Cloud-Native SQL
System Design ·

How Amazon Aurora Works Internally: The Log-Is-the-Database Architecture, Quorum Writes, and Storage-Compute Separation Behind Cloud-Native SQL

A deep dive into Aurora's storage-compute separation, the log-is-the-database design that pushes redo log processing to storage nodes, quorum-based replication across 6 copies in 3 AZs, protection group architecture, fast cloning, Serverless v2 scaling mechanics, and honest tradeoffs versus RDS, self-managed PostgreSQL, CockroachDB, and Cloud Spanner.

How CockroachDB Works Internally: Distributed SQL, Raft Consensus Per Range, and the Architecture Behind Serializable Transactions at Global Scale
System Design ·

How CockroachDB Works Internally: Distributed SQL, Raft Consensus Per Range, and the Architecture Behind Serializable Transactions at Global Scale

A deep dive into CockroachDB's internals: the 512MB range-based data model with automatic splitting, per-range Raft replication, MVCC timestamp ordering with hybrid logical clocks, DistSQL physical planning, serializable transactions with timestamp refreshes, closed timestamps for follower reads, online schema changes using multi-version schema descriptors, and production considerations for hotspots and write amplification.

How Container Runtimes Work Internally: Namespaces, Cgroups, and the OCI Stack From Docker Run to Process Isolation
System Design ·

How Container Runtimes Work Internally: Namespaces, Cgroups, and the OCI Stack From Docker Run to Process Isolation

Containers are not virtual machines and they are not magic. They are a thin composition of Linux kernel primitives: namespaces, cgroups, and a layered filesystem. This article traces exactly what happens between docker run and a running process, covering the OCI spec, containerd, runc, overlay filesystems, and container networking at the kernel level.

How YugabyteDB Works Internally: DocDB Storage, Tablet Splitting, and the Dual API Architecture That Scales PostgreSQL Horizontally
System Design ·

How YugabyteDB Works Internally: DocDB Storage, Tablet Splitting, and the Dual API Architecture That Scales PostgreSQL Horizontally

A deep dive into YugabyteDB internals covering the DocDB storage engine built on RocksDB LSM trees, per-tablet Raft consensus, YSQL and YCQL dual query layers, distributed MVCC with hybrid logical clocks, automatic tablet splitting and rebalancing, xCluster multi-region replication, and production considerations for hotspots, connection pooling, and schema design.