How Docker Works: Namespaces, Cgroups, Union Filesystems, and the Container Runtime from Build to Execution

When you run docker build and then docker run, two fundamentally different things happen. The build process assembles a layered filesystem from a sequence of instructions, each producing a content-addressable layer stored on disk. The run process takes that filesystem, wraps it in Linux kernel isolation primitives, enforces resource limits, and executes a process that believes it owns a machine. Most engineers understand neither step in enough detail to debug production failures or make good decisions about image design.

This article walks through both: the build pipeline from Dockerfile to image layers, the runtime pipeline from image to isolated process, and the production concerns that matter once you are running containers at scale.

The Image Format: Content-Addressable Layers

A Docker image is not a single file. It is a manifest that references a stack of compressed filesystem layers, each identified by the SHA-256 digest of its contents. The OCI Image Specification (the standard that Docker, containerd, and every registry implement) defines this layout explicitly.

When you pull an image, you are pulling:

1. A manifest (JSON), which lists ordered layer digests and points to an image config
2. An image config (JSON), which records metadata: environment variables, entrypoint, working directory, and the layer diff IDs
3. The layers themselves, each a gzipped tar archive of filesystem changes (additions, modifications, deletions)

The layer content is content-addressed: pull two images that share a base layer and Docker stores that layer once on disk. This is why docker pull node:20-alpine after already pulling node:20 with a shared base is fast: the shared layers are already present.

// Inspect image layer digests from the Docker API
import { execSync } from "child_process";

interface ImageLayer {
  digest: string;
  size: number;
}

function getImageLayers(image: string): ImageLayer[] {
  const raw = execSync(
    `docker manifest inspect ${image} --verbose 2>/dev/null || docker inspect ${image} --format '{{json .RootFS.Layers}}'`
  ).toString();

  // Using docker inspect for local images
  const layers = JSON.parse(
    execSync(`docker inspect ${image} --format '{{json .RootFS.Layers}}'`).toString().trim()
  ) as string[];

  return layers.map((digest, i) => ({ digest, size: i }));
}

const layers = getImageLayers("node:20-alpine");
layers.forEach((l, i) => console.log(`Layer ${i}: ${l.digest.slice(0, 19)}...`));

On disk, Docker’s overlay2 storage driver stores each layer as a directory under /var/lib/docker/overlay2/. Each directory contains a diff/ subdirectory with the filesystem changes for that layer, a link file with a short alias, and a lower file that lists the parent layer aliases. The overlay2 driver constructs the merged view by stacking these directories using the kernel’s OverlayFS.

OverlayFS: How Layers Become a Filesystem

OverlayFS merges multiple read-only directories (the lower layers) with one writable directory (the upper layer) into a single merged view. The kernel uses three directories internally:

• lowerdir: colon-separated list of read-only layer directories, ordered from newest to oldest
• upperdir: the writable container layer, created fresh for each container instance
• workdir: a scratch directory OverlayFS requires for atomic operations
• merged: the combined view presented to the container process

A read operation checks upperdir first, then walks lowerdir in order, returning the first match. A write to a file that exists only in a lower layer triggers a copy-on-write (COW): the kernel copies the entire file into upperdir, then applies the write. This is a critical performance consideration: writing a 100MB binary that lives in a lower layer incurs a 100MB copy before the first byte of your modification is written.

# See the exact OverlayFS mount options for a running container
CONTAINER_ID="$(docker ps -q --filter name=my-app)"
docker inspect "$CONTAINER_ID" \
  --format '{{.GraphDriver.Data.LowerDir}}' | tr ':' '\n' | head -5

# Directly observe the mount
CONTAINER_PID="$(docker inspect --format '{{.State.Pid}}' "$CONTAINER_ID")"
grep overlay /proc/"$CONTAINER_PID"/mounts

This COW behavior is why writing large temporary files inside a running container is expensive, and why Dockerfiles should avoid writing intermediate build artifacts to the container layer: each RUN instruction creates a new layer, and any files written but later deleted still exist in the intermediate layer, bloating the image.

The Build Process: Dockerfile Instruction Layering

Each Dockerfile instruction that modifies the filesystem produces a new image layer. Instructions that do not modify the filesystem (ENV, LABEL, EXPOSE, CMD, ENTRYPOINT) are recorded in the image config but do not produce separate layers.

The instructions that produce layers are: RUN, COPY, and ADD.

FROM node:20-alpine AS base
WORKDIR /app

# Layer 1: dependency manifests only (changes rarely)
COPY package.json package-lock.json ./

# Layer 2: install dependencies (rebuilds only when lockfile changes)
RUN npm ci --omit=dev

# Layer 3: application source (changes frequently)
COPY src/ ./src/

# Layer 4: compiled output
RUN npm run build

The ordering here is intentional and performance-critical. Understanding why requires understanding cache invalidation.

Build Cache Invalidation Rules

Docker’s build cache is content-based for COPY and ADD, and command-string-based for RUN.

For COPY and ADD: Docker computes a checksum of the files being copied. If any file in the source set has changed (by content, not mtime), the layer is invalidated and every subsequent layer is rebuilt from scratch.

For RUN: Docker compares the literal command string. If the string is identical to a previous build, the cached layer is used without actually executing the command, as long as no earlier layer was invalidated.

This creates two rules for writing efficient Dockerfiles:

Rule 1: Copy only what a step needs, as late as possible. Copying package.json and package-lock.json before npm ci means the install layer is only invalidated when dependencies change, not when source code changes.

Rule 2: Never rely on RUN cache for anything that has external dependencies. RUN apt-get update && apt-get install -y curl will use the cache even if the upstream package has changed. If you need freshness, use --no-cache at build time or pin versions explicitly.

// Detect cache invalidation risk: files copied before lockfile install
import { readFileSync } from "fs";

interface DockerfileAnalysis {
  instruction: string;
  lineNumber: number;
  cacheRisk: "high" | "low" | "none";
  reason: string;
}

function analyzeDockerfile(content: string): DockerfileAnalysis[] {
  const lines = content.split("\n");
  const results: DockerfileAnalysis[] = [];
  let seenInstall = false;

  lines.forEach((line, i) => {
    const trimmed = line.trim().toUpperCase();

    if (trimmed.startsWith("RUN") && line.includes("npm ci")) {
      seenInstall = true;
      results.push({
        instruction: line.trim(),
        lineNumber: i + 1,
        cacheRisk: "none",
        reason: "install step, cache is intentional here",
      });
    } else if (trimmed.startsWith("COPY") && !seenInstall) {
      const isMeta =
        line.includes("package.json") || line.includes("package-lock.json");
      results.push({
        instruction: line.trim(),
        lineNumber: i + 1,
        cacheRisk: isMeta ? "none" : "high",
        reason: isMeta
          ? "manifest-only copy, low invalidation frequency"
          : "source copy before install: invalidates install layer on every source change",
      });
    }
  });

  return results;
}

const dockerfile = readFileSync("/path/to/Dockerfile", "utf8");
const analysis = analyzeDockerfile(dockerfile);
analysis
  .filter((a) => a.cacheRisk === "high")
  .forEach((a) =>
    console.warn(`Line ${a.lineNumber}: ${a.reason}`)
  );

Multi-Stage Builds: Eliminating Build Tooling from the Runtime Image

Multi-stage builds solve a specific problem: the tools needed to compile or bundle an application (compilers, dev dependencies, build caches) have no place in the image that runs in production. Including them adds surface area for vulnerabilities and megabytes to every image pull and startup.

# Stage 1: full build environment
FROM node:20-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: runtime image, only what the process needs
FROM node:20-alpine AS runner
WORKDIR /app

# Create a non-root user before copying anything
RUN addgroup --system --gid 1001 appgroup && \
    adduser --system --uid 1001 --ingroup appgroup appuser

# Copy only compiled output and production dependencies
COPY --from=builder --chown=appuser:appgroup /app/dist ./dist
COPY --from=builder --chown=appuser:appgroup /app/node_modules ./node_modules
COPY --from=builder --chown=appuser:appgroup /app/package.json ./

USER appuser
EXPOSE 3000
CMD ["node", "dist/index.js"]

The runner stage has no TypeScript compiler, no source code, no dev dependencies. The layer count is minimal. Vulnerabilities in the build toolchain cannot be reached by a process that runs this image.

The Container Runtime Stack: containerd, runc, and the OCI Spec

When docker run is called, the actual execution path involves multiple components:

1. The Docker daemon (dockerd) receives the API call
2. dockerd delegates to containerd, a standalone container runtime daemon
3. containerd calls runc via the OCI Runtime Specification interface to create the container
4. runc calls into the Linux kernel to create namespaces and set up the container

The OCI Runtime Specification defines a JSON document called config.json that describes everything runc needs: the root filesystem path, namespace flags, cgroup paths, capabilities, seccomp profile, and the process to execute. runc is a thin binary. It reads this spec and makes syscalls.

# Extract the OCI runtime spec for a running container
CONTAINER_ID="$(docker ps -q --filter name=my-app)"
CONTAINER_PID="$(docker inspect --format '{{.State.Pid}}' "$CONTAINER_ID")"

# The spec is written here during container startup
cat /proc/"$CONTAINER_PID"/root/../../../containers/"$CONTAINER_ID"/config.json 2>/dev/null || \
  docker inspect "$CONTAINER_ID" --format '{{json .HostConfig}}' | jq .

This architecture means Docker is optional. You can run containers with containerd directly (using ctr or nerdctl), or with runc directly given a root filesystem and spec. Kubernetes has not required Docker since version 1.24: it talks directly to containerd via the Container Runtime Interface (CRI). Docker added a CRI-compatible shim (cri-dockerd) to remain usable as a Kubernetes runtime, but most production clusters moved to containerd natively.

Namespace Isolation: What Each Type Provides

Six namespace types underpin container isolation. Each controls visibility into a different resource class:

Namespace	Isolates	Production Impact
PID	Process IDs	PID 1 signal handling, no host process visibility
Network	Interfaces, routing, sockets	Each container gets its own eth0 via veth pair
Mount	Filesystem mount points	pivot_root into image rootfs
UTS	Hostname, domain name	Container hostname does not affect host
IPC	SysV IPC, POSIX message queues	No cross-container shared memory by default
User	UID/GID mappings	Root in container maps to unprivileged UID on host (rootless only)

The PID namespace has a sharp edge: the first process in the namespace is PID 1, and when PID 1 exits, the kernel sends SIGKILL to all other processes in the namespace immediately. Node.js does not handle signals the way a proper init process does. Running CMD ["node", "server.js"] as PID 1 means docker stop may not gracefully drain connections: the process gets SIGTERM, but if it does not call process.exit() explicitly after draining, the 10-second timeout fires and the container is killed. The fix is a minimal init like tini:

RUN apk add --no-cache tini
ENTRYPOINT ["/sbin/tini", "--"]
CMD ["node", "dist/index.js"]

tini runs as PID 1, forwards signals to the application, and reaps zombie processes. It adds under 1MB to the image.

Runtime Tradeoffs: Docker vs Podman vs containerd vs Kata

Runtime	Daemon Model	Root Required	OCI Compliant	Kubernetes CRI	Image Building	Security Model
Docker (dockerd)	Central daemon	Default yes, rootless optional	Yes (via containerd)	Via cri-dockerd shim	Native (Dockerfile)	Namespace + cgroup; user ns optional
Podman	Daemonless (fork/exec)	No (rootless default)	Yes (via runc/crun)	Via CRI-O compatible	Native	User namespaces by default
containerd	Daemon, lighter than dockerd	Yes	Yes (via runc)	Yes (native CRI)	Via BuildKit plugin	Namespace + cgroup
Kata Containers	Per-container microVM	Yes	Yes (OCI shim)	Yes (kata-agent)	Uses existing images	Hardware VM isolation

Podman’s daemonless model has a concrete benefit for security: there is no privileged long-running process. Each container is a direct child of the calling process, which means a vulnerability in the container runtime does not grant access to a daemon with root privileges. On developer machines, rootless Podman is the safer default. The tradeoff is slightly more complex volume permission handling, because UID mapping via user namespaces requires explicit configuration when host-mounted volumes need to be writable.

Kata Containers adds a lightweight VM between the container process and the host kernel. The OCI container spec is fulfilled, but instead of calling clone(2) to create namespaces, the runtime boots a stripped-down kernel (using QEMU or Firecracker) and runs the container inside it. The cost is startup latency (roughly 1-2 seconds versus 100-300ms for runc) and slightly higher memory overhead per container. The benefit is hard kernel isolation: a container escape would land in a microVM, not on the host.

Production Considerations

Image Size Optimization

Large images are a latency problem. A 1.2GB image on cold start adds 30-90 seconds to the first container start in a node that has not cached it. Three practices eliminate most of the bloat:

Use distroless or Alpine base images: gcr.io/distroless/nodejs20-debian12 contains only the Node.js runtime and its dependencies. No shell, no package manager, no utilities. This removes the entire class of vulnerabilities in those tools and shrinks the base from ~170MB (node:20) to ~115MB.

Clean up in the same RUN instruction: Each RUN is a layer. If you install a package manager, run it, then delete its cache in a separate RUN, the cache bytes are still present in the intermediate layer. Combine them:

RUN apt-get update && \
    apt-get install -y --no-install-recommends curl=7.88.1-10 && \
    rm -rf /var/lib/apt/lists/*

Use .dockerignore: A missing .dockerignore sends node_modules, .git, test fixtures, and local build artifacts into the build context. The build context is transmitted to the daemon before any instruction runs. On a large project, this alone can add 10-30 seconds to every build.

# .dockerignore
node_modules
.git
dist
coverage
*.test.ts
*.spec.ts
.env*

Security Hardening

Running containers as root is the default and the wrong choice. A process that escapes its container namespace while running as UID 0 on the host has full root access. Three hardening steps apply to almost every production workload:

Non-root user: Create an unprivileged user in the Dockerfile and switch to it with USER. This is already shown in the multi-stage example above.

Read-only root filesystem: Mount the container’s root filesystem as read-only. Any write attempt outside explicitly mounted writable volumes will fail. This eliminates an entire class of persistence attacks:

docker run --read-only \
  --tmpfs /tmp \
  --tmpfs /var/run \
  my-app:latest

In Kubernetes:

securityContext:
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1001

Seccomp profiles: Docker applies a default seccomp profile that blocks about 44 syscalls not needed by most applications. For higher assurance, generate a minimal profile using a tool like seccomp-gen or Falco to observe which syscalls your application actually uses, then block everything else. This limits what a compromised process can call into the kernel.

Container Startup Latency

Cold start latency for containers has three components: image pull time, layer extraction time, and process startup time. Only the third is constant. The first two scale with image size and whether the layers are already cached on the node.

On Kubernetes, the node image cache is populated by previous pulls. If a deployment scales to a new node, or a node is replaced, cold start time includes the full image pull. For latency-sensitive workloads, keep images small and use image prepulling (a DaemonSet that pulls images onto every node ahead of traffic) or snapshot-based startup (Firecracker snapshots, which can restore a process from memory state in under 200ms).

For Node.js specifically, startup time from node process launch to first request handled often dominates the measurable latency. Avoid importing large modules at startup that are only used on certain code paths:

// Lazy import for a module used only in a specific route
async function handlePdfExport(req: Request): Promise<Response> {
  // Only loaded when this handler runs, not at process start
  const { PDFDocument } = await import("pdf-lib");
  const doc = await PDFDocument.create();
  // ...
  return new Response(await doc.save());
}

For startup-critical paths, use --require preloading with explicit ordering and measure the startup timeline with node --prof or by logging timestamps at module load boundaries.

The Mental Model

Docker is a user interface around three independent mechanisms: the OCI image format (layered, content-addressed filesystem archives), the Linux kernel’s namespace and cgroup primitives (isolation and resource enforcement), and the OCI runtime interface (the contract between containerd and runc that says how to turn an image into a running process).

Understanding the split helps you debug correctly. A slow image build is almost always a cache invalidation problem rooted in COPY instruction ordering. An OOMKilled container is a cgroup memory.max enforcement event, readable from /sys/fs/cgroup/memory.events. A container that does not shut down gracefully is a PID 1 signal handling problem. A pod that takes 90 seconds to start on a new node is an image pull problem, not a startup problem.

The abstractions are clean. The primitives are stable. The failures are understandable once you know what layer they belong to.