How Containers Actually Work: Namespaces, Cgroups, and the Linux Primitives Behind Docker

When you run docker run --rm -it alpine sh, something happens in about 300 milliseconds that most engineers never look at closely. A new shell appears. It has its own process tree, its own network stack, its own filesystem root. It cannot see your host processes. Kill it and nothing leaks. This is not virtualization. There is no hypervisor, no guest kernel boot. It is the Linux kernel doing isolation work it has been capable of since 2008. Docker is, in a real sense, just a user-friendly wrapper around these primitives.

This article walks through exactly what those primitives are, how they compose into what we call a container, and what breaks in production when you misunderstand the boundaries.

The Illusion of Isolation

A container is not a VM. Both the host and every container running on it share one kernel. What changes between them is the kernel’s view of resources: which processes are visible, which network interfaces exist, which filesystem paths are accessible, and how much CPU and memory a group of processes can consume.

The kernel enforces these views through two independent mechanisms:

• Namespaces: What a process can see
• Cgroups: How much of a resource a process can use

Strip away the container runtime, the image format, the registry protocol, and what remains is a process that lives inside a set of namespaces and a cgroup hierarchy. That is the entire mechanism.

Namespaces

Linux currently has eight namespace types. Containers primarily rely on six of them. Each namespace type controls visibility into a specific resource class.

PID Namespace

A PID namespace creates an isolated process ID space. Processes inside the namespace see a fresh numbering starting at PID 1. From outside the namespace (the host), those processes are visible with their host PIDs, but they cannot see out.

// Node.js: demonstrate PID isolation by reading /proc/self/ns/pid
import { readFileSync } from "fs";
import { execSync } from "child_process";

function currentPidNamespace(): string {
  // /proc/self/ns/pid is a symlink whose target encodes the namespace inode
  return readFileSync("/proc/self/ns/pid").toString().trim();
}

function hostVisiblePids(): number[] {
  const output = execSync("ls /proc | grep -E '^[0-9]+$'").toString();
  return output
    .trim()
    .split("\n")
    .map(Number)
    .filter((n) => !isNaN(n));
}

console.log("Namespace inode:", currentPidNamespace());
console.log("Visible PIDs:", hostVisiblePids().length);

Inside a container, this script will show a small PID count and an inode that differs from the host. On the host it sees thousands. Same kernel, different namespace.

The first process in a new PID namespace becomes PID 1. This matters for signal handling: if PID 1 exits, the kernel sends SIGKILL to all other processes in the namespace. This is why container init processes (like tini) exist. A naive node server.js as PID 1 does not forward signals correctly, which causes docker stop to wait the full timeout before force-killing the container.

Network Namespace

A network namespace gives a process its own network stack: interfaces, routing tables, firewall rules, and socket space. By default, a new network namespace has only a loopback interface.

Docker creates a virtual Ethernet pair (veth) when starting a container. One end lives in the host network namespace, connected to a bridge (docker0). The other end lives inside the container’s network namespace, typically renamed to eth0. Packets flow between them through the kernel’s bridge forwarding logic.

# On the host, inspect Docker's bridge
ip link show docker0
# docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 ...

# List network namespaces (requires container to be running)
ls /var/run/docker/netns/

# Enter a container's network namespace without docker exec
CONTAINER_PID=$(docker inspect --format '{{.State.Pid}}' my-container)
nsenter --target "$CONTAINER_PID" --net ip addr

Kubernetes uses this same mechanism but delegates interface setup to CNI plugins. Flannel creates VXLAN tunnels between nodes. Calico uses BGP. Cilium replaces iptables with eBPF programs loaded directly into the network namespace’s ingress/egress hooks. The namespace boundary is the same across all of them.

Mount Namespace

A mount namespace controls which filesystem mounts a process sees. Processes in different mount namespaces can have entirely different views of the filesystem hierarchy. This is how a container gets its own root filesystem: the runtime creates a new mount namespace, pivot_root (or chroot) into the container image, and the process believes it is running on a complete Linux system.

import { execSync } from "child_process";

function listMounts(): string[] {
  return execSync("cat /proc/self/mountinfo")
    .toString()
    .trim()
    .split("\n");
}

// Inside a container, this will show the overlay mount as root
// and bind-mounted volumes as additional entries
const mounts = listMounts();
const overlayMount = mounts.find((line) => line.includes("overlay"));
console.log("Root filesystem mount:", overlayMount ?? "not overlay (bare metal?)");

Inside a Docker container, /proc/self/mountinfo shows an overlay filesystem as the root mount. On a host running bare Node.js, it shows the actual block device.

UTS, IPC, and User Namespaces

UTS namespace: Isolates hostname and domain name. Each container can have its own hostname (/etc/hostname) without affecting the host.

IPC namespace: Isolates System V IPC objects (message queues, semaphores, shared memory) and POSIX message queues. Containers cannot share IPC resources with other containers by default.

User namespace: Maps user IDs inside the namespace to different UIDs on the host. A process running as UID 0 (root) inside the container can be mapped to an unprivileged UID on the host. Docker does not enable user namespaces by default because they complicate volume ownership semantics, but rootless container runtimes (Podman, rootless Docker) rely on them for security.

Cgroups: Resource Enforcement

Namespaces control what a process sees. Cgroups control how much of the machine it can consume. Cgroups v2 (the unified hierarchy, default on most modern Linux distributions since kernel 5.2) organizes processes into a tree. Each node in the tree is a directory under /sys/fs/cgroup/. Interface files in those directories expose resource controllers.

The Controller Model

Controllers in cgroups v2 include cpu, memory, io, pids, and others. You enable them per-subtree by writing to cgroup.subtree_control.

# Create a cgroup for a batch job
mkdir /sys/fs/cgroup/batch-job

# Enable CPU and memory controllers
echo "+cpu +memory" > /sys/fs/cgroup/batch-job/cgroup.subtree_control

# Set a 200ms CPU quota per 1-second period (20% of one CPU)
echo "200000 1000000" > /sys/fs/cgroup/batch-job/cpu.max

# Set a 512MB memory hard limit
echo "$((512 * 1024 * 1024))" > /sys/fs/cgroup/batch-job/memory.max

# Set a soft memory target (above this, kernel starts reclaiming)
echo "$((256 * 1024 * 1024))" > /sys/fs/cgroup/batch-job/memory.high

# Move a process into the cgroup
echo $$ > /sys/fs/cgroup/batch-job/cgroup.procs

When a process exceeds memory.max, the kernel invokes the OOM killer for that cgroup. It kills processes within the cgroup, not across the system. This is the mechanism behind Kubernetes OOMKilled events.

// TypeScript: Read cgroup memory stats from within a container
import { readFileSync } from "fs";

interface MemoryStats {
  usageBytes: number;
  limitBytes: number;
  softLimitBytes: number;
  oomKillCount: number;
}

function readCgroupMemoryStats(): MemoryStats {
  // cgroups v2 paths used by containerd/Docker
  const basePath = "/sys/fs/cgroup";

  const usageBytes = parseInt(
    readFileSync(`${basePath}/memory.current`, "utf8").trim(),
    10
  );

  const limitRaw = readFileSync(`${basePath}/memory.max`, "utf8").trim();
  const limitBytes = limitRaw === "max" ? Infinity : parseInt(limitRaw, 10);

  const softLimitRaw = readFileSync(`${basePath}/memory.high`, "utf8").trim();
  const softLimitBytes =
    softLimitRaw === "max" ? Infinity : parseInt(softLimitRaw, 10);

  const eventsRaw = readFileSync(`${basePath}/memory.events`, "utf8");
  const oomKillLine = eventsRaw
    .split("\n")
    .find((line) => line.startsWith("oom_kill"));
  const oomKillCount = oomKillLine ? parseInt(oomKillLine.split(" ")[1], 10) : 0;

  return { usageBytes, limitBytes, softLimitBytes, oomKillCount };
}

const stats = readCgroupMemoryStats();
console.log(
  `Memory: ${(stats.usageBytes / 1024 / 1024).toFixed(1)}MB` +
    ` / ${stats.limitBytes === Infinity ? "unlimited" : (stats.limitBytes / 1024 / 1024).toFixed(1) + "MB"}`
);
console.log(`OOM kill count: ${stats.oomKillCount}`);

Running this inside a Kubernetes pod gives you the actual enforced limits from within the application. Useful as a startup health check or a readiness probe’s backing signal: if oomKillCount > 0 since process start, the pod has been OOM-killed before and is now running on a memory footprint that may be shrinking due to reclaim pressure.

CPU Throttling vs. OOM Kills

CPU and memory enforcement behave differently. A process that exceeds its memory limit is killed. A process that exceeds its CPU quota is throttled: the kernel simply stops scheduling it until the next quota period begins. This is silent. There is no log message. Your application just… runs slowly.

CPU throttling is one of the most common silent performance problems in Kubernetes. You can detect it by reading cpu.stat from within the container:

function readCpuThrottling(): { throttledPct: number } {
  const cpuStat = readFileSync("/sys/fs/cgroup/cpu.stat", "utf8");

  const parse = (key: string): number => {
    const line = cpuStat.split("\n").find((l) => l.startsWith(key));
    return line ? parseInt(line.split(" ")[1], 10) : 0;
  };

  const throttledUsec = parse("throttled_usec");
  const usageUsec = parse("usage_usec");

  const throttledPct =
    usageUsec > 0 ? (throttledUsec / (usageUsec + throttledUsec)) * 100 : 0;

  return { throttledPct };
}

const { throttledPct } = readCpuThrottling();
if (throttledPct > 10) {
  console.warn(`CPU throttled ${throttledPct.toFixed(1)}% of wall time`);
}

Anything above 5-10% throttle ratio is worth investigating. At 30%+, you will see latency spikes in p99 metrics. The fix is either raising the CPU limit, reducing the CPU request-to-limit ratio, or identifying the processing burst causing the spike.

Union Filesystems: How Images Become Containers

Container images are not tarballs you extract and run. They are stacked layers composed at runtime into a single filesystem view via a union filesystem. Docker’s default storage driver on modern Linux is OverlayFS (overlay2).

OverlayFS works by merging two directories:

• lower layer(s): the read-only image layers, stacked from base to most-recent
• upper layer: a writable layer created fresh for each container instance

When a process reads a file, the kernel checks the upper layer first. If the file exists there, that version is returned. If not, it falls through to the lower layers in order. When a process writes to a file that exists only in a lower layer, the kernel performs a copy-on-write: the file is copied up to the upper layer and the write is applied there. The lower layer is never modified.

This is why docker commit is fast (the upper layer diff is already computed), why pulling a new image with shared base layers is fast (lower layers already cached), and why writing large files inside a container incurs a one-time copy-on-write cost.

# Inspect the overlay mounts of a running container
CONTAINER_ID=$(docker ps -q --filter name=my-app)
docker inspect "$CONTAINER_ID" --format '{{.GraphDriver.Data}}'

# Output (approximate):
# map[LowerDir:/var/lib/docker/overlay2/abc123/diff:/var/lib/docker/overlay2/def456/diff
#      MergedDir:/var/lib/docker/overlay2/xyz789/merged
#      UpperDir:/var/lib/docker/overlay2/xyz789/diff
#      WorkDir:/var/lib/docker/overlay2/xyz789/work]

MergedDir is what the container process sees as its root. UpperDir accumulates writes. LowerDir lists all image layers separated by colons, deepest base first.

How a Container Runtime Wires This Together

When docker run executes, the flow is roughly:

1. The Docker daemon (or containerd directly) pulls or validates the image layers and assembles the OverlayFS merge.
2. runc (the OCI-compliant container runtime) is invoked with a runtime spec derived from the image config.
3. runc calls clone(2) with namespace flags: CLONE_NEWPID | CLONE_NEWNET | CLONE_NEWNS | CLONE_NEWUTS | CLONE_NEWIPC. This creates the new namespaces.
4. Inside the new namespaces, runc calls pivot_root(2) to change the filesystem root to the OverlayFS merge directory.
5. runc writes the container’s PID into the appropriate cgroup under /sys/fs/cgroup/ to enforce resource limits.
6. The container’s entrypoint process executes. From its perspective, it is running on a fresh Linux system.

The host kernel sees this process with its host PID, its host-visible cgroup path, and its namespace inodes. Nothing is hidden from the host. The isolation is purely a different view of shared kernel state.

Tradeoffs: Container Isolation Approaches

Approach	Kernel Shared	Isolation Strength	Overhead	Cold Start	Use Case
Linux containers (runc)	Yes	Namespace + cgroup	Minimal	~100-500ms	Standard workloads
gVisor (runsc)	Partial (syscall interception)	Syscall sandboxing	10-20% CPU	~500ms	Untrusted code, multi-tenant
Kata Containers	No (separate kernel)	Full VM-level	~5-10% vs VM	~1-2s	Regulated workloads, PCI/HIPAA
Firecracker microVMs	No (separate kernel)	Full VM-level	Very low	~125ms	Serverless function isolation
Rootless containers (Podman)	Yes	Namespace + user ns	Minimal	~100-500ms	Dev environments, no-root hosts
Bare metal process	Yes	None	Zero	Immediate	Trusted single-tenant services

Standard runc containers are the right default. The namespace and cgroup boundary is sufficient for most threat models. gVisor makes sense when you run untrusted user-submitted workloads (think CI runners, notebook execution). Kata or Firecracker are appropriate when compliance requirements demand hardware-level isolation or when a kernel vulnerability in shared-kernel containers would be catastrophic.

Production Considerations: Debugging Resource Limits in Kubernetes

Most container resource limit failures in Kubernetes fall into three categories: OOMKilled, CPU throttling, and PID limit exhaustion. Here is the debugging workflow for each.

OOMKilled

# Check termination reason on recently dead pods
kubectl get pods -A | grep OOMKilled

# Get the last terminated container's reason and exit code
kubectl get pod my-pod -o jsonpath='{.status.containerStatuses[0].lastState.terminated}'

# Output: {"exitCode":137,"reason":"OOMKilled","startedAt":"...","finishedAt":"..."}

Exit code 137 is SIGKILL (128 + 9). When you see it with reason: OOMKilled, the container’s cgroup hit its memory limit. Your next step is to understand what was using the memory. For Node.js:

// Expose a /metrics endpoint with heap and RSS
import { createServer } from "http";
import { readFileSync } from "fs";

function cgroupMemoryUsage(): number {
  try {
    return parseInt(
      readFileSync("/sys/fs/cgroup/memory.current", "utf8").trim(),
      10
    );
  } catch {
    return 0;
  }
}

createServer((req, res) => {
  if (req.url !== "/metrics") {
    res.end();
    return;
  }

  const mem = process.memoryUsage();
  const cgroupBytes = cgroupMemoryUsage();

  res.setHeader("Content-Type", "text/plain");
  res.end(
    [
      `nodejs_heap_used_bytes ${mem.heapUsed}`,
      `nodejs_heap_total_bytes ${mem.heapTotal}`,
      `nodejs_rss_bytes ${mem.rss}`,
      `nodejs_external_bytes ${mem.external}`,
      `cgroup_memory_current_bytes ${cgroupBytes}`,
    ].join("\n") + "\n"
  );
}).listen(9090);

The gap between rss and heapUsed often reveals off-heap allocations: native addons, Buffer pools, or mapped files. The cgroup value includes all memory attributed to the container’s cgroup, including shared library mappings. If cgroup_memory_current_bytes is significantly higher than nodejs_rss_bytes, you are likely looking at multiple processes in the same cgroup or a native module’s off-heap allocation.

CPU Throttling in Kubernetes

Set CPU limits conservatively and you will throttle. Set them too high and you create noisy neighbor risk in a shared cluster. The right approach is to measure first:

// Periodically log throttle ratio, structured for log-based alerting
import { readFileSync } from "fs";

function getCpuStats(): Record<string, number> {
  const raw = readFileSync("/sys/fs/cgroup/cpu.stat", "utf8");
  const stats: Record<string, number> = {};
  for (const line of raw.trim().split("\n")) {
    const [key, val] = line.split(" ");
    stats[key] = parseInt(val, 10);
  }
  return stats;
}

let previousStats = getCpuStats();

setInterval(() => {
  const current = getCpuStats();

  const deltaUsage = current.usage_usec - previousStats.usage_usec;
  const deltaThrottled = current.throttled_usec - previousStats.throttled_usec;
  const throttlePct =
    deltaUsage + deltaThrottled > 0
      ? (deltaThrottled / (deltaUsage + deltaThrottled)) * 100
      : 0;

  console.log(
    JSON.stringify({
      level: throttlePct > 10 ? "warn" : "info",
      event: "cpu_throttle_sample",
      throttle_pct: parseFloat(throttlePct.toFixed(2)),
      throttled_periods: current.nr_throttled - previousStats.nr_throttled,
    })
  );

  previousStats = current;
}, 15_000);

Log this to your observability stack. Alert when throttle_pct exceeds 10% over a 5-minute window. Correlate with request latency spikes. In most cases, either the CPU limit in the pod spec is too tight (raise it) or a batch operation is stealing scheduling time from the request path (refactor to a separate worker or use setImmediate yielding).

PID Limit Exhaustion

Less common, but it happens: a runaway process spawning child processes fills the container’s PID namespace. Kubernetes defaults to no PID limit at the container level unless --pid-limit is configured at the node level.

# Check pod PID count against the node's pid.max
kubectl exec my-pod -- cat /sys/fs/cgroup/pids.current
kubectl exec my-pod -- cat /sys/fs/cgroup/pids.max

If pids.current is approaching pids.max, the next fork(2) call will return EAGAIN. In Node.js this surfaces as spawn EAGAIN errors from child_process. Set an explicit podSpec.containers[].resources.limits for PIDs in your admission webhook or OPA policy to prevent this from quietly degrading a node.

The Mental Model That Matters

A container is a Linux process (or process tree) with a carefully constructed view of the world. It sees a narrow slice of the kernel’s resource state: its own PIDs, its own network stack, its own filesystem. The kernel enforces resource consumption limits through cgroup controllers. The image format provides a layered, copy-on-write filesystem assembled at runtime.

When something goes wrong in production, the debugging path follows the primitives directly. OOMKilled traces to memory.max in the cgroup. CPU throttling traces to cpu.max. Filesystem surprises trace to OverlayFS copy-on-write semantics. Network failures trace to CNI-managed virtual ethernet pairs and namespace routing tables.

Understanding what Docker actually does when you type docker run means you can debug production failures without cargo-culting restart strategies, and it means you can reason clearly about what Kubernetes resource limits actually enforce at the kernel level.