Designing a Code Execution Sandbox: Container Isolation, Resource Limits, and Secure Multi-Tenant Code Running at Scale

Most teams building code execution systems underestimate the threat model. They reach for Docker, set a memory limit, add a timeout, and call it done. That works until a tenant crafts a payload that escapes the container, exhausts the kernel’s PID table, or opens a raw socket to exfiltrate data. The gap between “runs in a container” and “safe to run arbitrary untrusted code” is wider than it looks.

This article covers the full architecture of a production code execution sandbox: isolation levels and their real security boundaries, resource limiting with cgroups and ulimits, the execution lifecycle from submission to output capture, multi-tenant scheduling with warm pools, and output streaming. The focus is practical: each decision comes with tradeoffs that matter in production.

Isolation Levels and Their Actual Security Boundaries

The first architectural decision is how much you trust your isolation layer. Container runtimes share a kernel. That matters a great deal when the code you’re running is adversarial.

Docker containers give you namespace and cgroup isolation. Processes inside can’t see the host filesystem (when configured correctly) or other containers’ processes. But they share the host kernel. A kernel exploit running inside a Docker container can compromise the host. For an online judge running student code or a developer playground with authenticated users, this is often acceptable. For running arbitrary public submissions or AI agent tool calls that reach the internet, it is not.

gVisor interposes a user-space kernel between the container process and the host kernel. System calls from the container are intercepted and handled by gVisor’s “Sentry” component, which implements a subset of the Linux syscall interface. The host kernel surface area exposed to tenant code shrinks dramatically. The cost is performance: syscall-heavy workloads (network I/O, file I/O) are significantly slower. CPU-bound compute sees less overhead.

Firecracker microVMs give you a real VM boundary. Each execution runs in a separate KVM-based virtual machine with its own kernel. The host kernel is not shared. Boot time is around 125ms for a minimal image, which is fast enough for many use cases when combined with warm pool pre-provisioning. Firecracker is the isolation layer behind AWS Lambda.

V8 Isolates operate at the language runtime level. A V8 isolate is an independent JavaScript heap with no shared memory with other isolates. This is how Cloudflare Workers achieves sub-millisecond startup: no new process, no container, just a new heap. The tradeoff is that you’re limited to JavaScript (or languages that compile to it), and the isolation guarantee depends entirely on the V8 runtime having no exploitable bugs. V8 isolates are appropriate when you control the language but not when you need to run arbitrary binaries.

WebAssembly offers portable sandboxing at the instruction level. Wasm modules run in a memory-safe linear memory space with no direct access to host resources. They can only call host functions explicitly imported. This makes Wasm a strong isolation primitive for plugins, user-defined functions, and multi-language sandboxing where you can compile target languages to Wasm.

Resource Limiting

Isolation stops lateral movement between tenants. Resource limiting stops a single tenant from consuming resources that affect others or the host.

CPU limiting via cgroups v2 uses the cpu.max interface: 500000 1000000 means the cgroup can use 500ms of CPU time in every 1000ms period, which is a 50% CPU quota. For per-execution limits, set this at container start time. The Go runtime, JVM, and Node.js all respect cgroup CPU quotas when configured correctly (some versions require explicit flags).

Memory limiting via memory.max in cgroups v2 sets a hard limit. When the process exceeds it, the OOM killer terminates it. Set memory.swap.max to 0 to prevent swap usage, which could otherwise allow a tenant to cause significant I/O load on the host. Always set both.

File descriptor limits via ulimit -n prevent a process from opening thousands of sockets or files. Combined with network namespace isolation, you can prevent all outbound network access entirely, which is correct for most sandboxed execution scenarios.

Execution timeout is not a cgroup feature. You enforce it externally: start the execution process, start a timer, send SIGKILL after the timeout expires. Do not use SIGTERM for untrusted code since it can be caught. SIGKILL cannot. Add a watchdog process outside the sandbox that holds a timer and issues the kill.

interface ResourceLimits {
  cpuQuotaMs: number;      // CPU ms allowed per period
  cpuPeriodMs: number;     // cgroup cpu.max period
  memoryLimitMb: number;   // hard memory cap
  maxOpenFiles: number;    // ulimit -n
  timeoutMs: number;       // wall-clock execution timeout
  networkAccess: boolean;  // whether to allow outbound network
}

async function applyLimits(
  containerId: string,
  limits: ResourceLimits
): Promise<void> {
  // Set cgroup cpu.max
  const cpuMax = `${limits.cpuQuotaMs * 1000} ${limits.cpuPeriodMs * 1000}`;
  await writeCgroupFile(containerId, "cpu.max", cpuMax);

  // Set memory hard limit and disable swap
  await writeCgroupFile(
    containerId,
    "memory.max",
    String(limits.memoryLimitMb * 1024 * 1024)
  );
  await writeCgroupFile(containerId, "memory.swap.max", "0");
}

The Execution Lifecycle

A well-designed execution lifecycle has six distinct stages, each with clear failure modes.

1. Submission and validation. Accept the code, language, and input. Validate the payload size, character encoding, and that the requested language is supported. Reject early before consuming any execution resources.

2. Queue routing. Route the submission to an appropriate execution queue. Different queue lanes handle different priority levels: interactive playground requests need sub-second latency, batch judge runs can tolerate seconds. A fair queue prevents any single tenant from monopolizing execution workers.

3. Sandbox provisioning. Either pull a warm sandbox from a pool or create a new one. Write the code and input into the sandbox. For container-based approaches, this means mounting a read-only volume with the source file. For Firecracker, it means configuring the rootfs before boot.

4. Execution. Start the process inside the sandbox with limits applied. Attach to stdout and stderr streams. Start the watchdog timer.

5. Output capture. Collect stdout, stderr, exit code, and resource usage (wall time, CPU time, peak memory). Cap output size to prevent runaway output from filling buffers.

6. Cleanup. Stop the sandbox, reset state, and return it to the warm pool or destroy it. This step must run even if execution fails. Use try/finally everywhere.

interface ExecutionRequest {
  submissionId: string;
  language: string;
  sourceCode: string;
  stdin: string;
  limits: ResourceLimits;
}

interface ExecutionResult {
  submissionId: string;
  stdout: string;
  stderr: string;
  exitCode: number;
  wallTimeMs: number;
  cpuTimeMs: number;
  peakMemoryMb: number;
  status: "accepted" | "tle" | "mle" | "runtime_error" | "system_error";
}

class SandboxRunner {
  private pool: SandboxPool;

  async execute(request: ExecutionRequest): Promise<ExecutionResult> {
    const sandbox = await this.pool.acquire(request.language);
    const start = Date.now();

    try {
      await sandbox.writeCode(request.sourceCode, request.stdin);
      await applyLimits(sandbox.id, request.limits);

      const proc = await sandbox.start();
      const result = await this.waitWithTimeout(proc, request.limits.timeoutMs);

      return {
        submissionId: request.submissionId,
        stdout: result.stdout.slice(0, 64 * 1024), // cap at 64KB
        stderr: result.stderr.slice(0, 16 * 1024),
        exitCode: result.exitCode,
        wallTimeMs: Date.now() - start,
        cpuTimeMs: result.cpuTimeMs,
        peakMemoryMb: result.peakMemoryMb,
        status: this.classify(result),
      };
    } finally {
      await this.pool.release(sandbox);
    }
  }

  private async waitWithTimeout(
    proc: SandboxProcess,
    timeoutMs: number
  ): Promise<ProcessResult> {
    const timeout = new Promise<never>((_, reject) =>
      setTimeout(() => reject(new Error("TLE")), timeoutMs)
    );
    return Promise.race([proc.wait(), timeout]);
  }
}

Multi-Tenant Scheduling and Warm Pool Management

Cold start latency is the enemy of interactive use cases. Spinning up a new Docker container takes 300-800ms. Booting a Firecracker microVM takes ~125ms. Neither is acceptable for a playground where users expect sub-200ms response times.

The solution is a warm pool: pre-booted sandboxes waiting for submissions. The pool manager keeps N sandboxes alive per language, ready to accept code. When a sandbox is claimed, a replacement is provisioned in the background. When a sandbox completes execution and is returned, it is either reset and returned to the pool (for stateless execution) or destroyed and replaced.

Reset fidelity matters. For Docker containers, “reset” means killing the process and clearing any written files. The container itself stays running. For Firecracker, the VM must be snapshotted before first use and restored from snapshot on each reset. Firecracker’s snapshot/restore path is around 8ms, which makes warm-pool restore viable.

Fair queuing prevents starvation. Implement per-tenant rate limiting at the queue level: a tenant submitting 1000 jobs should not block another tenant’s interactive request. Two-lane queuing with an interactive lane (low latency, low throughput cap) and a batch lane (higher throughput, best-effort latency) covers most use cases.

interface PoolConfig {
  language: string;
  minWarm: number;
  maxWarm: number;
  idleTimeoutMs: number;
}

class SandboxPool {
  private available: Map<string, Sandbox[]> = new Map();
  private pending: Map<string, Promise<Sandbox>> = new Map();

  async acquire(language: string): Promise<Sandbox> {
    const pool = this.available.get(language) ?? [];

    if (pool.length > 0) {
      const sandbox = pool.pop()!;
      this.available.set(language, pool);
      // Replenish in background
      this.replenish(language).catch(() => {});
      return sandbox;
    }

    // No warm sandbox available, provision on demand
    return this.provision(language);
  }

  async release(sandbox: Sandbox): Promise<void> {
    const pool = this.available.get(sandbox.language) ?? [];
    const config = this.getConfig(sandbox.language);

    if (pool.length < config.maxWarm) {
      await sandbox.reset();
      pool.push(sandbox);
      this.available.set(sandbox.language, pool);
    } else {
      await sandbox.destroy();
    }
  }

  private async replenish(language: string): Promise<void> {
    const config = this.getConfig(language);
    const pool = this.available.get(language) ?? [];

    while (pool.length < config.minWarm) {
      const sandbox = await this.provision(language);
      pool.push(sandbox);
    }

    this.available.set(language, pool);
  }
}

Output Streaming with SSE

Batch execution returns results after completion. Interactive playgrounds benefit from real-time output streaming: users see stdout as the program runs, not after it finishes.

Server-Sent Events (SSE) is the right transport for this. It is unidirectional (server to client), works over HTTP/1.1, and does not require a persistent WebSocket upgrade. The execution worker streams chunks as they arrive from the sandbox’s stdout pipe.

import { Hono } from "hono";
import { streamSSE } from "hono/streaming";

const app = new Hono();

app.get("/execute/:submissionId/stream", async (c) => {
  const submissionId = c.req.param("submissionId");

  return streamSSE(c, async (stream) => {
    const sandbox = await runner.getActiveSandbox(submissionId);
    if (!sandbox) {
      await stream.writeSSE({ event: "error", data: "not_found" });
      return;
    }

    for await (const chunk of sandbox.stdoutChunks()) {
      await stream.writeSSE({
        event: "stdout",
        data: Buffer.from(chunk).toString("base64"),
      });
    }

    const result = await sandbox.waitForExit();
    await stream.writeSSE({
      event: "done",
      data: JSON.stringify({
        exitCode: result.exitCode,
        wallTimeMs: result.wallTimeMs,
        status: result.status,
      }),
    });
  });
});

Pipe the sandbox’s stdout through a line buffer and emit each line as a separate SSE event. Cap the output size at the server side before the data reaches the client. A runaway print loop should terminate via the output cap, not by exhausting the connection’s buffer.

Production Considerations

Seccomp profiles limit which system calls a process inside a container can make. Docker applies a default seccomp profile that blocks around 44 syscalls. For a code execution sandbox, you want a stricter custom profile that blocks networking syscalls (socket, connect, sendmsg) entirely when network access is disabled, as well as dangerous system management calls (mount, pivot_root, ptrace). Write the profile to match exactly what the target runtime needs and nothing more.

The PID limit matters. Without a PID limit, a fork bomb inside a container can exhaust the host’s PID table and crash unrelated processes. Set pids.max in the cgroup before execution starts.

Output encoding is a common bug. Untrusted code can emit arbitrary bytes. Store and transmit output as base64 or byte arrays. Do not assume UTF-8. Decoding on the client side after display is safer than trying to sanitize binary output at the sandbox boundary.

Filesystem isolation requires read-only mounts everywhere except a small /tmp directory with a size limit. The tmpfs mount with size=32m is the right pattern: it is in-memory, size-capped, and destroyed with the container.

Audit logging should record every execution: tenant ID, language, start time, end time, exit code, and resource consumption. This is not just for billing; it is your primary forensic tool when a sandbox behaves unexpectedly.

Isolation Tradeoffs

Approach	Isolation Strength	Cold Start	Throughput	Kernel Exposure	Best For
Docker (runc)	Medium	300-800ms	High	Full host kernel	Trusted tenants, dev environments
gVisor (runsc)	High	400-900ms	Medium	Minimal via Sentry	Untrusted code, light syscall workloads
Firecracker microVM	Very High	125ms (restore 8ms)	Medium	None (separate VM kernel)	Public submissions, serverless platforms
V8 Isolates	High (JS only)	Sub-1ms	Very High	None (runtime-mediated)	JavaScript-only, edge functions
WebAssembly	High (portable)	1-10ms	High	None (linear memory)	Plugin systems, multi-language UDFs

The right choice depends on your threat model. An internal developer playground for authenticated employees can use Docker with a strict seccomp profile. A public online judge running untrusted code from the internet needs Firecracker or gVisor. An edge function platform that needs sub-millisecond startup with JavaScript workloads is a natural fit for V8 isolates. A plugin system that needs to support multiple languages with strong isolation at low overhead is where Wasm shines.

The common mistake is choosing an isolation level based on familiarity rather than threat model. Docker is familiar. It is not always sufficient. Know what your tenants are, what your attack surface is, and size the isolation accordingly. Then enforce resource limits as a separate, independent layer so that even a sandboxed process that behaves pathologically cannot affect its neighbors.