Back-Pressure in Distributed Systems: Flow Control Patterns That Prevent Cascading Overload

A producer that generates work faster than a consumer can process it is one of the most reliable ways to take down a distributed system. Not through a dramatic crash, but through a slow accumulation of in-flight work that eventually exhausts memory, threads, file descriptors, or all three at once.

The failure pattern is predictable. An upstream service sends requests at 10,000 per second. A downstream service can sustain 6,000 per second. The gap gets absorbed by a queue. The queue grows. Memory climbs. The garbage collector starts spending more time collecting than executing. Latency rises. The upstream service, not seeing errors, keeps sending. Eventually the downstream service either OOM-kills or starts responding with timeouts. The upstream service retries. Load doubles. Everything falls over together.

Back-pressure is the mechanism that prevents this. Instead of absorbing excess load into an ever-growing buffer, you signal the producer to slow down, drop excess work at a defined boundary, or shed load before it enters the system. The specific strategy depends on your latency requirements, data durability guarantees, and the nature of the producer-consumer relationship.

This article covers the four core strategies, TypeScript implementations for each, back-pressure at the message queue layer, back-pressure at the HTTP layer, reactive streams, and the observability primitives that tell you when back-pressure is actually doing its job.

Why Unbounded Queues Are a Trap

The intuitive response to a producer-consumer imbalance is to add a buffer. If the consumer is temporarily slow, let work queue up and process it when things calm down. This works for transient bursts. It fails for sustained overload.

An unbounded queue hides the problem. The producer thinks everything is fine because its sends are succeeding. The consumer is buried under a growing backlog that it will never catch up on. By the time the queue size triggers an alert, you are already in recovery mode, not prevention mode.

Bounded queues surface the problem at the boundary rather than hiding it. When the queue is full, the system is forced to make a decision: reject new work, block the producer, or drop the oldest item. That forced decision is the back-pressure mechanism.

Memory is the other constraint. A queue of 1 million pending HTTP request bodies can consume gigabytes of heap. A Node.js process that exceeds its heap limit exits immediately. A Java service that exceeds its configured heap starts GC-pausing into the seconds range before the OOM killer arrives. Explicit bounds keep memory footprint predictable.

The Four Core Strategies

1. Drop

The simplest approach: when a bounded buffer is full, discard new arrivals. No blocking, no signaling, no complex coordination. Producers are not slowed down; excess work simply disappears.

class DroppingQueue<T> {
  private readonly items: T[] = [];

  constructor(private readonly capacity: number) {}

  enqueue(item: T): boolean {
    if (this.items.length >= this.capacity) {
      return false; // dropped
    }
    this.items.push(item);
    return true;
  }

  dequeue(): T | undefined {
    return this.items.shift();
  }

  get size(): number {
    return this.items.length;
  }
}

// Usage: track drop rate as a metric
const queue = new DroppingQueue<WorkItem>(1000);
let droppedCount = 0;

function submit(item: WorkItem): void {
  const accepted = queue.enqueue(item);
  if (!accepted) {
    droppedCount++;
    metrics.increment('queue.dropped');
  }
}

Drop is appropriate when individual items are cheap to discard: telemetry events, non-critical notifications, log aggregation samples. It is not appropriate when every item must be processed, such as financial transactions or user-visible operations.

A variant is tail-drop (discard newest, keep oldest) versus head-drop (discard oldest, keep newest). Head-drop is useful when stale work is worthless: if you are processing real-time sensor readings and the queue is full, you want the latest reading, not a reading from 30 seconds ago.

2. Buffer with Bounds

Accept work up to a fixed limit, then block the producer or return an error that the producer can act on. Unlike drop, the producer knows its item was not accepted.

class BoundedAsyncQueue<T> {
  private readonly items: T[] = [];
  private readonly waiters: Array<{
    resolve: (item: T) => void;
  }> = [];
  private readonly pendingEnqueues: Array<{
    item: T;
    resolve: () => void;
    reject: (err: Error) => void;
  }> = [];

  constructor(private readonly capacity: number) {}

  async enqueue(item: T, timeoutMs = 5000): Promise<void> {
    if (this.waiters.length > 0) {
      // A consumer is already waiting; deliver directly
      const waiter = this.waiters.shift()!;
      waiter.resolve(item);
      return;
    }

    if (this.items.length < this.capacity) {
      this.items.push(item);
      return;
    }

    // Queue is full: block until space is available or timeout
    return new Promise<void>((resolve, reject) => {
      const timer = setTimeout(() => {
        const idx = this.pendingEnqueues.findIndex((e) => e.resolve === resolve);
        if (idx !== -1) this.pendingEnqueues.splice(idx, 1);
        reject(new Error('Enqueue timeout: queue full'));
      }, timeoutMs);

      this.pendingEnqueues.push({
        item,
        resolve: () => {
          clearTimeout(timer);
          resolve();
        },
        reject,
      });
    });
  }

  async dequeue(): Promise<T> {
    if (this.items.length > 0) {
      const item = this.items.shift()!;
      this.drainPending();
      return item;
    }

    return new Promise<T>((resolve) => {
      this.waiters.push({ resolve });
    });
  }

  private drainPending(): void {
    if (this.pendingEnqueues.length > 0 && this.items.length < this.capacity) {
      const pending = this.pendingEnqueues.shift()!;
      this.items.push(pending.item);
      pending.resolve();
    }
  }
}

The enqueue timeout is important. Without it, a stalled consumer will block producers indefinitely. With it, producers get a clear error they can handle: retry, circuit-break, or return an error to their own caller.

3. Signal Upstream

Rather than blocking or dropping, explicitly tell the producer to reduce its rate. This is the most cooperative form of back-pressure and requires the producer to be controllable. It shows up in reactive streams (RxJS, Node.js readable streams), token bucket rate limiters, and explicit feedback channels between services.

interface BackPressureSignal {
  pause(): void;
  resume(): void;
}

class ControlledConsumer {
  private paused = false;
  private readonly processingQueue: unknown[] = [];
  private readonly MAX_QUEUE = 500;

  constructor(private readonly source: BackPressureSignal) {}

  onData(chunk: unknown): void {
    this.processingQueue.push(chunk);

    if (this.processingQueue.length >= this.MAX_QUEUE && !this.paused) {
      this.paused = true;
      this.source.pause(); // signal back to producer
    }
  }

  private async processNext(): Promise<void> {
    while (true) {
      const item = this.processingQueue.shift();
      if (item === undefined) break;

      await this.process(item);

      if (this.paused && this.processingQueue.length < this.MAX_QUEUE / 2) {
        this.paused = false;
        this.source.resume(); // hysteresis: resume at 50% capacity
      }
    }
  }

  private async process(item: unknown): Promise<void> {
    // actual processing logic
  }
}

// Node.js readable stream back-pressure
import { Readable, Writable } from 'stream';

function pipeWithBackPressure(source: Readable, sink: Writable): void {
  source.on('data', (chunk) => {
    const canContinue = sink.write(chunk);
    if (!canContinue) {
      source.pause(); // built-in back-pressure signal
      sink.once('drain', () => source.resume());
    }
  });
}

The hysteresis threshold (resume at 50% rather than 0%) is not optional. Without it, you get oscillation: pause at 500 items, resume at 499, pause again immediately. A gap between the pause and resume thresholds creates a stable operating range.

4. Adaptive Rate

Instead of binary pause/resume, the producer continuously adjusts its throughput based on feedback from the consumer. This is what TCP congestion control does: it adjusts the send window based on ACKs and packet loss signals.

class AdaptiveRateLimiter {
  private currentRatePerSec: number;
  private readonly minRate: number;
  private readonly maxRate: number;
  private lastSendTime = Date.now();
  private tokenBucket: number;

  constructor(options: {
    initialRate: number;
    minRate: number;
    maxRate: number;
  }) {
    this.currentRatePerSec = options.initialRate;
    this.minRate = options.minRate;
    this.maxRate = options.maxRate;
    this.tokenBucket = options.initialRate;
  }

  onSuccess(): void {
    // Additive increase
    this.currentRatePerSec = Math.min(
      this.currentRatePerSec * 1.1,
      this.maxRate
    );
  }

  onBackPressure(consumerQueueDepth: number, consumerCapacity: number): void {
    const utilization = consumerQueueDepth / consumerCapacity;
    if (utilization > 0.8) {
      // Multiplicative decrease on overload
      this.currentRatePerSec = Math.max(
        this.currentRatePerSec * 0.5,
        this.minRate
      );
    }
  }

  async acquire(): Promise<void> {
    const now = Date.now();
    const elapsed = (now - this.lastSendTime) / 1000;
    this.tokenBucket = Math.min(
      this.currentRatePerSec,
      this.tokenBucket + elapsed * this.currentRatePerSec
    );
    this.lastSendTime = now;

    if (this.tokenBucket >= 1) {
      this.tokenBucket -= 1;
      return;
    }

    const waitMs = ((1 - this.tokenBucket) / this.currentRatePerSec) * 1000;
    await new Promise((resolve) => setTimeout(resolve, waitMs));
    this.tokenBucket = 0;
  }
}

The additive-increase/multiplicative-decrease (AIMD) shape is intentional. It probes gently upward but backs off hard on overload signals, which prevents oscillation at high utilization.

Back-Pressure in Message Queues

Kafka: Consumer Lag as the Signal

Kafka does not push messages to consumers. Consumers pull at their own pace, which means the broker never gets overwhelmed by a slow consumer. The back-pressure is implicit: if the consumer is slow, it just processes fewer messages per second.

The signal you need to watch is consumer lag: the difference between the latest offset on a partition and the last committed offset of a consumer group. Lag of zero means the consumer is keeping up. Sustained lag growth means it is not.

import { Kafka } from 'kafkajs';

const kafka = new Kafka({ brokers: ['localhost:9092'] });
const admin = kafka.admin();

async function getConsumerLag(
  groupId: string,
  topic: string
): Promise<{ partition: number; lag: bigint }[]> {
  await admin.connect();

  const [offsets, topicOffsets] = await Promise.all([
    admin.fetchOffsets({ groupId, topics: [topic] }),
    admin.fetchTopicOffsets(topic),
  ]);

  const groupOffsets = offsets[0].partitions;
  const latestOffsets = topicOffsets;

  return groupOffsets.map((groupPartition) => {
    const latest = latestOffsets.find(
      (p) => p.partition === groupPartition.partition
    );
    const currentOffset = BigInt(groupPartition.offset);
    const latestOffset = BigInt(latest?.offset ?? '0');
    return {
      partition: groupPartition.partition,
      lag: latestOffset - currentOffset,
    };
  });
}

// Control concurrency to bound in-flight processing
const consumer = kafka.consumer({ groupId: 'my-group' });
await consumer.connect();
await consumer.subscribe({ topic: 'orders', fromBeginning: false });

await consumer.run({
  partitionsConsumedConcurrently: 3, // bound concurrent partition processing
  eachMessage: async ({ message }) => {
    // processing here; Kafka waits for this promise before fetching more
    await processOrder(message);
  },
});

eachMessage provides natural back-pressure: Kafka’s client library only fetches the next batch after the current message resolves. If processOrder is slow, fetch rate drops automatically. partitionsConsumedConcurrently bounds how many partitions are processed in parallel.

max.poll.interval.ms is the Kafka-side enforcement: if a consumer does not call poll within the interval, Kafka assumes it is dead and triggers a rebalance. Keep your message processing time well under this limit, or increase it deliberately.

RabbitMQ: Prefetch as the Lever

RabbitMQ is a push model, which means the broker will send messages as fast as the consumer can receive them over TCP. Without a bound, a slow consumer ends up with thousands of unacknowledged messages sitting in its process memory.

basicQos (prefetch count) is the control. It tells RabbitMQ to send at most N unacknowledged messages to a consumer at any time. Until the consumer acks one, no new message is delivered.

import amqp from 'amqplib';

const connection = await amqp.connect('amqp://localhost');
const channel = await connection.createChannel();

// Critical: set prefetch before consuming
await channel.prefetch(10); // at most 10 unacked messages in flight

await channel.consume('work-queue', async (msg) => {
  if (!msg) return;

  try {
    await processMessage(msg.content);
    channel.ack(msg);
  } catch (err) {
    // Requeue on transient failure; dead-letter on permanent failure
    const requeue = isTransient(err);
    channel.nack(msg, false, requeue);
  }
});

Prefetch of 1 gives maximum back-pressure but minimum throughput: each message must be acked before the next arrives. Prefetch of 100 gives higher throughput but more unacknowledged work in flight. A value between 5 and 20 is a reasonable starting point for most workloads; tune based on measured throughput and memory usage.

SQS: Visibility Timeout as Implicit Back-Pressure

SQS is a pull model. Your consumers poll for messages. The visibility timeout controls how long a message is hidden from other consumers after it is received. If your consumer does not delete the message within the timeout, it becomes visible again and another consumer picks it up.

This is not back-pressure in the traditional sense, but the pattern for managing throughput is to control your polling concurrency and batch size.

import { SQSClient, ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';

const sqs = new SQSClient({ region: 'us-east-1' });
const QUEUE_URL = process.env.QUEUE_URL!;
const CONCURRENCY = 5; // bound parallel processing

async function processWithBackPressure(): Promise<void> {
  const inFlight = new Set<Promise<void>>();

  while (true) {
    // Only fetch more work when we have capacity
    if (inFlight.size >= CONCURRENCY) {
      await Promise.race(inFlight);
      continue;
    }

    const response = await sqs.send(
      new ReceiveMessageCommand({
        QueueUrl: QUEUE_URL,
        MaxNumberOfMessages: Math.min(10, CONCURRENCY - inFlight.size),
        WaitTimeSeconds: 20, // long polling; avoids busy-wait
        VisibilityTimeout: 60,
      })
    );

    for (const message of response.Messages ?? []) {
      const task = processAndDelete(message).finally(() => {
        inFlight.delete(task);
      });
      inFlight.add(task);
    }
  }
}

async function processAndDelete(message: { Body?: string; ReceiptHandle?: string }): Promise<void> {
  await processMessage(message.Body!);
  await sqs.send(
    new DeleteMessageCommand({
      QueueUrl: QUEUE_URL,
      ReceiptHandle: message.ReceiptHandle!,
    })
  );
}

Bounding inFlight.size is the back-pressure mechanism. The poller only requests more messages when it has capacity to process them, preventing unbounded accumulation in process memory.

Back-Pressure at the HTTP Layer

429 with Retry-After

When a service is overloaded, returning HTTP 429 with a Retry-After header is the correct signaling mechanism. It tells the caller: I am at capacity, try again in N seconds. Well-behaved clients will back off and retry; poorly-behaved clients will hammer anyway (which is where rate limiting enforcement matters).

import Fastify from 'fastify';

const app = Fastify();

const REQUEST_LIMIT = 100;
let inFlightRequests = 0;

app.addHook('onRequest', async (request, reply) => {
  if (inFlightRequests >= REQUEST_LIMIT) {
    reply
      .status(429)
      .header('Retry-After', '5')
      .send({ error: 'Too many requests', retryAfterSeconds: 5 });
    return;
  }
  inFlightRequests++;
});

app.addHook('onResponse', async () => {
  inFlightRequests--;
});

app.get('/process', async (request, reply) => {
  const result = await doWork();
  return result;
});

Tracking in-flight requests rather than a sliding-window request count captures the resource contention model more accurately. If each request holds a database connection, the number of simultaneous in-flight requests determines connection pool pressure, not the requests-per-second rate.

Load Shedding

Load shedding is the decision to reject low-priority work to protect capacity for high-priority work. It is a more deliberate version of dropping: you classify incoming requests and shed the cheap ones first.

type Priority = 'critical' | 'standard' | 'background';

function getPriority(request: { headers: Record<string, string> }): Priority {
  const token = request.headers['x-priority'];
  if (token === 'critical') return 'critical';
  if (token === 'background') return 'background';
  return 'standard';
}

function shouldShed(priority: Priority, currentLoad: number): boolean {
  // Shed background at 70% load, standard at 90%, never shed critical
  if (priority === 'background' && currentLoad > 0.7) return true;
  if (priority === 'standard' && currentLoad > 0.9) return true;
  return false;
}

function getCurrentLoad(): number {
  return inFlightRequests / REQUEST_LIMIT;
}

The priority classification must come from a trusted source (internal headers, authenticated tokens) or anyone can mark their requests as critical.

Reactive Streams and Async Iterators

Node.js readable streams have built-in back-pressure via the write() return value and the drain event, as shown earlier. Async iterators generalize this pattern without the push/pull impedance mismatch.

async function* generateWork(): AsyncGenerator<WorkItem> {
  let cursor = 0;
  while (true) {
    const batch = await fetchBatch(cursor, 100);
    if (batch.length === 0) return;
    for (const item of batch) {
      yield item;
    }
    cursor += batch.length;
  }
}

async function consumeWithBackPressure(): Promise<void> {
  for await (const item of generateWork()) {
    // The generator is paused here until we're done processing
    // No messages accumulate in memory beyond what we've pulled
    await processItem(item);
  }
}

The for await...of loop is a pull model: the generator only executes until the next yield, then pauses. The consumer drives the pace. There is no buffer between producer and consumer beyond the single item being processed. For workloads where each item is expensive and you want zero accumulation, this is the cleanest pattern.

For parallel processing with bounded concurrency, combine async iterators with a semaphore.

class Semaphore {
  private permits: number;
  private readonly queue: Array<() => void> = [];

  constructor(permits: number) {
    this.permits = permits;
  }

  async acquire(): Promise<void> {
    if (this.permits > 0) {
      this.permits--;
      return;
    }
    await new Promise<void>((resolve) => this.queue.push(resolve));
    this.permits--;
  }

  release(): void {
    this.permits++;
    const next = this.queue.shift();
    if (next) {
      this.permits--;
      next();
    }
  }
}

async function parallelConsume<T>(
  source: AsyncIterable<T>,
  concurrency: number,
  handler: (item: T) => Promise<void>
): Promise<void> {
  const sem = new Semaphore(concurrency);
  const tasks: Promise<void>[] = [];

  for await (const item of source) {
    await sem.acquire();
    const task = handler(item).finally(() => sem.release());
    tasks.push(task);
  }

  await Promise.all(tasks);
}

Observability for Back-Pressure

Back-pressure is invisible unless you instrument it. The metrics that matter:

Queue depth: the number of items waiting to be processed. Alert at 70% of your bounded capacity, page at 90%. A queue that is always near capacity means your consumer is under-provisioned.

Consumer lag (Kafka-specific): alert when lag exceeds N minutes of production at peak rate. This gives you time to scale before the lag becomes unrecoverable.

Drop rate and shed rate: how many items per second are being dropped or shed. A non-zero drop rate is not always an emergency (it might be expected for telemetry), but a rising drop rate on critical workloads is.

Producer block time: how long producers wait before a queue slot opens. High block time indicates consumer throughput is genuinely insufficient.

In-flight request count: the active concurrency of your service. If this regularly approaches your limit, you are operating with thin headroom.

// Minimal instrumentation example (prometheus-compatible counters)
const metrics = {
  queueDepth: gauge('queue_depth'),
  itemsDropped: counter('queue_items_dropped_total'),
  producerBlockMs: histogram('queue_producer_block_ms'),
  inFlight: gauge('requests_in_flight'),
};

// Record before enqueue attempt
const start = Date.now();
const accepted = queue.enqueue(item);
metrics.producerBlockMs.observe(Date.now() - start);

if (!accepted) {
  metrics.itemsDropped.inc({ reason: 'queue_full' });
} else {
  metrics.queueDepth.set(queue.size);
}

Production Tradeoffs

Strategy	Latency impact	Data loss risk	Producer complexity	Best for
Drop (tail)	None	High	None	Telemetry, sampling
Drop (head)	None	Medium	None	Real-time streams, sensor data
Bounded buffer + block	Medium (blocks producer)	None	Low	Batch jobs, internal pipelines
Signal upstream (pause/resume)	Low	None	Medium	Streaming, reactive pipelines
Adaptive rate (AIMD)	Low	None	High	Cross-service, long-running producers
HTTP 429 + Retry-After	Low	Caller-dependent	None (HTTP standard)	Public APIs, inter-service HTTP
Load shedding	None	Medium	Medium	Mixed-priority traffic

The latency impact of bounded buffer blocking depends entirely on how long the consumer is backed up. In the worst case, it is unbounded. If your upstream caller has a tight timeout, a blocking enqueue that takes longer than that timeout turns into a different kind of failure. Pair bounded blocking with enqueue timeouts as shown in the implementation above.

Where Back-Pressure Fails

Back-pressure is not a substitute for capacity planning. It handles transient overload and protects against burst spikes, but if your sustained production rate exceeds your sustained consumption capacity, back-pressure just makes the failure mode cleaner, not avoidable.

Retry amplification is a related trap. When producers receive 429 or enqueue timeouts, they retry. If every caller retries immediately, the retry traffic is at least as large as the original load that caused the overload. Exponential backoff with jitter on the producer side is mandatory; otherwise back-pressure signals cause retry storms.

Finally, back-pressure signals can propagate incorrectly in async call chains. If service A sends to service B, which is backed up and blocks, service A’s thread or async context is now held. A’s own concurrency is consumed waiting for B. Back-pressure needs to be considered end-to-end, not just at individual queue boundaries.

A system that handles back-pressure correctly degrades gracefully: it processes less, it is explicit about what it dropped, and it recovers without manual intervention when the load subsides. That is the goal, and every strategy here is a different tradeoff toward it.