Web Engineering ·

Web Workers in Production: Offloading Computation, Shared Memory, and Communication Patterns in TypeScript

How to use Web Workers effectively in production TypeScript applications, covering structured cloning vs transferable objects, SharedArrayBuffer, Comlink, worker pooling, and real benchmarks for image processing, CSV parsing, and cryptographic operations.

Web Workers in Production: Offloading Computation, Shared Memory, and Communication Patterns in TypeScript

The main thread is a single-lane road. Every frame budget, every user interaction, every layout recalculation runs in that same lane. When you drop a 200ms computation in the middle of it, users feel it: the scroll stutters, the button doesn’t respond, the animation freezes. The browser does not hide this from you. requestAnimationFrame stops firing. The input handler queues up. You get a jank report.

Web Workers have existed since 2009, yet most production applications still run expensive computation on the main thread. The typical reason is friction: the messaging API is verbose, TypeScript types are awkward across the thread boundary, and debugging is harder. This article removes those objections and shows how to use Workers correctly in TypeScript, from basic message passing through shared memory, worker pooling, and production testing.

When to Offload to a Worker

Not everything belongs in a worker. The question is whether the work is CPU-bound and whether it takes longer than roughly 5ms (the time budget for a 60fps frame).

Move to a worker:

  • CSV or JSON parsing of files over ~500KB
  • Image manipulation: resizing, filtering, format conversion
  • Cryptographic operations: hashing, key derivation, encryption
  • Data transformations: sorting large arrays, tree traversal, compression
  • Wasm-heavy computation

Keep on the main thread:

  • DOM manipulation (Workers cannot touch the DOM)
  • Anything that needs window or document
  • Short operations under ~2ms (the overhead of postMessage is real)
  • Work that requires tight synchronization with UI state

A concrete benchmark to calibrate this: parsing a 5MB CSV file takes roughly 280ms on the main thread in Chrome. That is 16 dropped frames. Running the same operation in a dedicated worker takes 290ms wall-clock time but zero main-thread impact. The user interaction remains responsive.

Structured Cloning vs Transferable Objects

When you send data between the main thread and a worker, the browser serializes it. This uses the structured clone algorithm, which is effectively a deep copy. For a 10MB ArrayBuffer, that copy costs around 15ms.

Transferable objects skip the copy by transferring ownership. The original reference becomes neutered (unusable) in the sender, and the data is owned by the receiver with no copy cost. This matters for ArrayBuffer, MessagePort, ImageBitmap, and OffscreenCanvas.

// Structured clone: data is copied, original remains usable
const buffer = new ArrayBuffer(10_000_000);
worker.postMessage({ buffer }); // ~15ms copy

// Transferable: data ownership moves, original becomes detached
const buffer = new ArrayBuffer(10_000_000);
worker.postMessage({ buffer }, [buffer]); // ~0ms transfer
console.log(buffer.byteLength); // 0 — buffer is now detached

For image processing, this matters significantly. An uncompressed 4K image (3840 * 2160 * 4 bytes) is around 32MB. Copying that on every frame is not viable; transferring it is.

// main.ts
async function processFrame(imageData: ImageData): Promise<ImageData> {
  const buffer = imageData.data.buffer;

  return new Promise((resolve) => {
    worker.onmessage = (e: MessageEvent<{ buffer: ArrayBuffer; width: number; height: number }>) => {
      const result = new ImageData(
        new Uint8ClampedArray(e.data.buffer),
        e.data.width,
        e.data.height
      );
      resolve(result);
    };

    // Transfer the buffer — main thread no longer owns it
    worker.postMessage(
      { buffer, width: imageData.width, height: imageData.height },
      [buffer]
    );
  });
}
// image-worker.ts
self.onmessage = (e: MessageEvent<{ buffer: ArrayBuffer; width: number; height: number }>) => {
  const { buffer, width, height } = e.data;
  const data = new Uint8ClampedArray(buffer);

  // Apply grayscale filter in-place
  for (let i = 0; i < data.length; i += 4) {
    const luminance = 0.299 * data[i] + 0.587 * data[i + 1] + 0.114 * data[i + 2];
    data[i] = data[i + 1] = data[i + 2] = luminance;
  }

  // Transfer back — worker no longer owns it
  self.postMessage({ buffer, width, height }, [buffer]);
};

SharedArrayBuffer and Atomics

For scenarios where two threads need to read and write shared state without copying, SharedArrayBuffer provides a memory region accessible from both. Unlike transferables, neither side loses access. Both threads see the same physical memory.

This introduces actual concurrent memory access, which means you need Atomics to avoid data races. Atomics.load, Atomics.store, Atomics.add, and Atomics.compareExchange are the primitives.

SharedArrayBuffer requires two HTTP headers to be set (Cross-Origin Isolation):

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Without these headers, SharedArrayBuffer is undefined in modern browsers. This is a security requirement introduced after Spectre.

// Shared ring buffer for high-throughput audio processing
interface RingBufferHeader {
  writePos: number; // index 0
  readPos: number;  // index 1
}

const HEADER_SIZE = 2; // two Int32 slots for read/write positions
const BUFFER_SIZE = 1024;

function createRingBuffer(): SharedArrayBuffer {
  return new SharedArrayBuffer((HEADER_SIZE + BUFFER_SIZE) * Float32Array.BYTES_PER_ELEMENT);
}

// Worker writes audio samples
function writeAudioSamples(shared: SharedArrayBuffer, samples: Float32Array): boolean {
  const header = new Int32Array(shared, 0, HEADER_SIZE);
  const data = new Float32Array(shared, HEADER_SIZE * 4);

  const writePos = Atomics.load(header, 0);
  const readPos = Atomics.load(header, 1);
  const available = BUFFER_SIZE - ((writePos - readPos + BUFFER_SIZE) % BUFFER_SIZE);

  if (available < samples.length) return false; // buffer full

  for (let i = 0; i < samples.length; i++) {
    data[(writePos + i) % BUFFER_SIZE] = samples[i];
  }

  Atomics.store(header, 0, (writePos + samples.length) % BUFFER_SIZE);
  Atomics.notify(header, 0, 1); // wake any waiting reader
  return true;
}

SharedArrayBuffer is the right tool for high-frequency shared state: audio worklets, game loops, real-time data visualization where copying 60 times per second is too expensive. For most other cases, message passing with transferables is simpler and sufficient.

The raw postMessage API is functional but verbose. You end up maintaining message type enums, manual routing switches, and callback management. Comlink wraps this into a proxy that looks like a regular async function call.

npm install comlink
// heavy-worker.ts
import { expose } from 'comlink';

const api = {
  async parseCSV(csvText: string): Promise<Record<string, string>[]> {
    const lines = csvText.split('\n');
    const headers = lines[0].split(',');

    return lines.slice(1)
      .filter(line => line.trim())
      .map(line => {
        const values = line.split(',');
        return Object.fromEntries(headers.map((h, i) => [h.trim(), values[i]?.trim() ?? '']));
      });
  },

  async hashPassword(password: string, iterations = 100_000): Promise<string> {
    const encoder = new TextEncoder();
    const keyMaterial = await crypto.subtle.importKey(
      'raw',
      encoder.encode(password),
      'PBKDF2',
      false,
      ['deriveBits']
    );

    const bits = await crypto.subtle.deriveBits(
      {
        name: 'PBKDF2',
        salt: encoder.encode('static-salt-replace-with-random'),
        iterations,
        hash: 'SHA-256'
      },
      keyMaterial,
      256
    );

    return Array.from(new Uint8Array(bits))
      .map(b => b.toString(16).padStart(2, '0'))
      .join('');
  }
};

expose(api);
export type HeavyWorkerApi = typeof api;
// main.ts
import { wrap } from 'comlink';
import type { HeavyWorkerApi } from './heavy-worker';

const worker = new Worker(new URL('./heavy-worker.ts', import.meta.url), { type: 'module' });
const api = wrap<HeavyWorkerApi>(worker);

// Looks like a regular async function call
const rows = await api.parseCSV(csvText);
const hash = await api.hashPassword(userInput);

Comlink handles the postMessage round-trip, error propagation (rejected promises surface correctly), and type inference. The tradeoff is that Comlink does not support transferable objects in the same ergonomic way. For data under a few megabytes, this is fine. For large buffers, you may need to drop back to raw postMessage with explicit transfer lists.

Worker Pooling

A single worker processes tasks serially. If you have concurrent work (multiple users uploading files simultaneously, for example), one worker creates a queue. A worker pool distributes work across N workers and keeps all of them busy.

// worker-pool.ts
import { wrap, Remote } from 'comlink';
import type { HeavyWorkerApi } from './heavy-worker';

type WorkerEntry = {
  worker: Worker;
  api: Remote<HeavyWorkerApi>;
  busy: boolean;
};

export class WorkerPool {
  private workers: WorkerEntry[] = [];
  private queue: Array<() => void> = [];

  constructor(private size: number = navigator.hardwareConcurrency ?? 4) {
    for (let i = 0; i < this.size; i++) {
      const worker = new Worker(new URL('./heavy-worker.ts', import.meta.url), { type: 'module' });
      this.workers.push({
        worker,
        api: wrap<HeavyWorkerApi>(worker),
        busy: false
      });
    }
  }

  async run<T>(task: (api: Remote<HeavyWorkerApi>) => Promise<T>): Promise<T> {
    const entry = this.workers.find(w => !w.busy);

    if (!entry) {
      // All workers busy — wait for a free one
      await new Promise<void>(resolve => this.queue.push(resolve));
      return this.run(task);
    }

    entry.busy = true;
    try {
      return await task(entry.api);
    } finally {
      entry.busy = false;
      this.queue.shift()?.();
    }
  }

  terminate(): void {
    this.workers.forEach(({ worker }) => worker.terminate());
  }
}

// Usage
const pool = new WorkerPool(4);

// These four tasks run in parallel across four workers
const results = await Promise.all([
  pool.run(api => api.parseCSV(csv1)),
  pool.run(api => api.parseCSV(csv2)),
  pool.run(api => api.hashPassword(p1)),
  pool.run(api => api.hashPassword(p2)),
]);

Set pool size to navigator.hardwareConcurrency as a starting point, but cap it. A machine with 16 logical cores does not benefit from 16 workers doing I/O-light CPU work due to context switching overhead. In practice, 4 to 8 workers covers most use cases.

Communication Patterns Tradeoffs

PatternThroughputLatencyComplexityWhen to Use
postMessage (structured clone)MediumLowLowSmall data, infrequent calls
postMessage (transferable)HighLowMediumLarge buffers, ArrayBuffer ownership transfer
SharedArrayBuffer + AtomicsVery highNear-zeroHighShared state, audio worklets, game loops
Comlink (proxy)MediumLowLowErgonomic API, typical data sizes
Worker pool + ComlinkHighLowMediumConcurrent tasks, multi-user workloads
BroadcastChannelLowLowLowOne-to-many worker coordination

Production Examples

CSV Parsing

A 5MB CSV with 100K rows takes approximately 280ms on the main thread. In a worker, it takes 290ms wall-clock but zero main-thread time. For a data import UI, this means the progress bar animates smoothly instead of freezing.

The key is reading the file as text() on the main thread (fast) then transferring the string to the worker. Strings are structured-cloned, not transferable, but even a 5MB string copies in under 5ms.

Image Processing

For user-uploaded images that need resizing before upload, OffscreenCanvas is the right approach. Create an OffscreenCanvas in the worker, draw to it with drawImage, then export via convertToBlob.

// resize-worker.ts
import { expose } from 'comlink';

const api = {
  async resize(
    imageBitmap: ImageBitmap,
    targetWidth: number,
    targetHeight: number
  ): Promise<Blob> {
    const canvas = new OffscreenCanvas(targetWidth, targetHeight);
    const ctx = canvas.getContext('2d')!;
    ctx.drawImage(imageBitmap, 0, 0, targetWidth, targetHeight);
    return canvas.convertToBlob({ type: 'image/webp', quality: 0.85 });
  }
};

expose(api);
// main.ts — transfer ImageBitmap to avoid copying pixel data
const bitmap = await createImageBitmap(file);
const blob = await workerApi.resize(transfer(bitmap), 800, 600);

Note transfer(bitmap) from Comlink: this tells Comlink to transfer the ImageBitmap rather than clone it.

Cryptographic Operations

PBKDF2 with 100K iterations takes 200-400ms depending on hardware. Running this on the main thread locks up the UI while the user waits for login. In a worker it is transparent. The Web Crypto API is available in workers, so no external library is needed.

For client-side AES-GCM encryption of large files, the worker processes chunks of 64KB at a time and posts progress messages back to the main thread. The main thread only handles progress UI updates.

Testing Workers in TypeScript

Testing across the thread boundary requires a different approach than unit tests. Workers do not exist in Node.js, so you have two options.

Option 1: Extract the pure logic, test it directly. The worker file imports business logic from a shared module. Test that module directly without any worker involvement.

// csv-parser.ts — pure logic, no worker-specific APIs
export function parseCSV(text: string): Record<string, string>[] {
  const lines = text.split('\n');
  const headers = lines[0].split(',');
  return lines.slice(1)
    .filter(line => line.trim())
    .map(line => {
      const values = line.split(',');
      return Object.fromEntries(headers.map((h, i) => [h.trim(), values[i]?.trim() ?? '']));
    });
}

// heavy-worker.ts — just wires up the logic
import { expose } from 'comlink';
import { parseCSV } from './csv-parser';
expose({ parseCSV });

Unit test csv-parser.ts directly. No mocking needed.

Option 2: Integration test with a real browser (Playwright or Vitest browser mode). For production confidence, run a small integration test that spins up the worker and exercises the full message-passing path.

// worker.integration.test.ts (Vitest browser mode)
import { describe, it, expect } from 'vitest';
import { wrap } from 'comlink';
import type { HeavyWorkerApi } from './heavy-worker';

describe('HeavyWorker', () => {
  it('parses CSV in worker thread', async () => {
    const worker = new Worker(new URL('./heavy-worker.ts', import.meta.url), { type: 'module' });
    const api = wrap<HeavyWorkerApi>(worker);

    const result = await api.parseCSV('name,age\nAlice,30\nBob,25');
    expect(result).toEqual([
      { name: 'Alice', age: '30' },
      { name: 'Bob', age: '25' }
    ]);

    worker.terminate();
  });
});

Production Considerations

Worker startup cost. Creating a worker takes 5-50ms depending on the browser and script size. For workers that run on demand (user clicks “import CSV”), initialize the worker eagerly on page load and keep it alive rather than creating it per task.

Error handling. Unhandled errors in workers do not propagate to the main thread automatically. With raw postMessage, you must listen to worker.onerror. Comlink propagates errors as rejected promises, which is easier to handle. Either way, instrument worker errors explicitly.

worker.onerror = (event: ErrorEvent) => {
  console.error('Worker error:', event.message, event.filename, event.lineno);
  // Send to your error tracking service
};

Memory pressure. Workers have their own heap. A pool of 8 workers each holding a 100MB buffer is 800MB of memory. Profile with Chrome DevTools Memory tab for worker threads. Terminate workers when they are no longer needed.

Module workers and bundling. { type: 'module' } in the Worker constructor enables ES module syntax inside the worker. Bundlers (Vite, esbuild, webpack) handle new Worker(new URL('./worker.ts', import.meta.url)) correctly for code splitting. Always use the URL constructor form, not a bare string path, for reliable bundler support.

SharedArrayBuffer availability. Even with the correct COOP/COEP headers, some CDNs strip them. Verify at runtime:

if (typeof SharedArrayBuffer === 'undefined') {
  // Fall back to postMessage with cloning
  console.warn('SharedArrayBuffer unavailable — check COOP/COEP headers');
}

The Real Return

Workers are not about raw speed. The computation takes the same amount of time. The return is responsiveness: the main thread stays free to handle input, run animations, and paint frames while the work happens in parallel. For operations over 50ms, this difference is perceptible to users. For operations over 200ms, it is the difference between a working UI and a frozen one.

The patterns here compose well. Start with Comlink for ergonomics, add transferables when data size demands it, introduce a worker pool when concurrency matters, and reach for SharedArrayBuffer only when shared state at high frequency is genuinely required. Most production use cases stop at the second step.

More in Web Engineering

How Rspack Works Internally: Webpack-Compatible Module Graph, Rust Compilation Pipeline, and the Incremental Build Architecture Behind the Fastest Webpack Replacement
Web Engineering ·

How Rspack Works Internally: Webpack-Compatible Module Graph, Rust Compilation Pipeline, and the Incremental Build Architecture Behind the Fastest Webpack Replacement

A deep dive into Rspack's Rust-based architecture, module graph construction, SWC transformation pipeline, webpack compatibility layer, incremental compilation, and what the tradeoffs look like for teams migrating from webpack.

How Turbopack Works Internally: Incremental Computation, Function-Level Caching, and the Rust-Based Bundler Architecture Behind Next.js
Web Engineering ·

How Turbopack Works Internally: Incremental Computation, Function-Level Caching, and the Rust-Based Bundler Architecture Behind Next.js

A deep dive into Turbopack's architecture covering the Turbo engine's incremental computation model, function-level caching, Rust-based module resolution, granular HMR invalidation, SWC integration, persistent caching, and how it compares to webpack, Vite, esbuild, and Rspack.

How gRPC Works: Protocol Buffers, HTTP/2 Multiplexing, and Bidirectional Streaming From Service Definition to Wire Format
Web Engineering ·

How gRPC Works: Protocol Buffers, HTTP/2 Multiplexing, and Bidirectional Streaming From Service Definition to Wire Format

A deep dive into gRPC internals: Protocol Buffer IDL and the buf codegen pipeline, HTTP/2 stream multiplexing and HPACK header compression, all four RPC patterns with TypeScript, channel management, deadline propagation, interceptors, load balancing from pick-first to xDS, the health checking protocol, and production tradeoffs vs REST and GraphQL.

How Node.js Works Internally: The Event Loop, libuv Thread Pool, and Async I/O Architecture From require() to Process Exit
Web Engineering ·

How Node.js Works Internally: The Event Loop, libuv Thread Pool, and Async I/O Architecture From require() to Process Exit

A deep dive into Node.js internals covering V8 JIT compilation, the six-phase libuv event loop, thread pool mechanics, microtask queue priority, async/await desugaring, CommonJS versus ESM module resolution, and production tuning considerations with TypeScript examples.