Container Security in Production: Image Scanning, Runtime Policies, and Supply Chain Integrity

Most container security failures are not zero-days. They are misconfigurations shipped through the same pipeline that ships features: a node:latest base image carrying 200 known CVEs, a container running as root because no one added a USER directive, a secret injected as an environment variable and visible in docker inspect. These are the failure modes worth addressing first, and they are all fixable before a single line of application code runs.

This article covers the full container security lifecycle: image construction, vulnerability scanning in CI/CD, runtime enforcement, supply chain integrity, and secrets handling. Each section includes concrete examples. The tradeoffs table near the end covers the friction each control introduces, so you can make informed decisions about sequencing.

---

Building Secure Images

The attack surface of a container starts with its base image. Every package in the image is a potential exploit path. Minimal base images reduce that surface without requiring any runtime work.

Start with a minimal base

node:20-alpine ships roughly 30 packages. node:20 (Debian) ships roughly 400. For most Node.js applications, the Debian variant adds nothing useful and a lot of CVE exposure.

For the absolute minimum, use distroless images from Google’s gcr.io/distroless registry. These images contain only the runtime and its dependencies, with no shell, no package manager, and no apt. You cannot exec into them, which is a security property, not a limitation.

# Multi-stage build: build stage uses full Node, runtime stage is distroless
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM gcr.io/distroless/nodejs20-debian12 AS runtime
WORKDIR /app
# Copy only what the runtime needs
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
EXPOSE 3000
CMD ["dist/server.js"]

Multi-stage builds are the mechanism that makes this practical. Build tools, test dependencies, and intermediate artifacts stay in the builder stage and never reach the final image. The runtime image contains only what executes in production.

Run as a non-root user

By default, Docker containers run as root (UID 0) unless you specify otherwise. Running as root inside a container does not grant host root access in normal configurations, but it does in privilege-escalation exploit chains and when volume mounts are involved. The fix is one line:

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

FROM node:20-alpine AS runtime
# Create a non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
WORKDIR /app
COPY --from=builder --chown=appuser:appgroup /app/dist ./dist
COPY --from=builder --chown=appuser:appgroup /app/node_modules ./node_modules
USER appuser
EXPOSE 3000
CMD ["node", "dist/server.js"]

The --chown flag on COPY ensures the files are owned by the non-root user before the USER directive switches context. If you forget --chown, the process cannot read its own files.

Eliminate unnecessary capabilities

Linux capabilities let you grant specific privileges without full root. Most web applications need zero capabilities. Drop them all and add back only what you actually need:

# In Kubernetes, this is enforced at the pod spec level
# In Docker run directly:
# docker run --cap-drop ALL --cap-add NET_BIND_SERVICE myimage

In Kubernetes, set this in the pod security context:

securityContext:
  runAsNonRoot: true
  runAsUser: 1000
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL
  readOnlyRootFilesystem: true

readOnlyRootFilesystem: true is worth calling out specifically. It means the container filesystem is immutable at runtime. If a process tries to write to /tmp or anywhere outside an explicitly mounted volume, it gets an error. This catches a class of attacks that try to modify binaries or write persistent tooling after initial compromise.

---

Vulnerability Scanning in CI/CD

Scanning should happen at two points: when an image is built, and on a schedule against images already deployed. CVEs are published continuously, so an image that was clean at build time may have known vulnerabilities within weeks.

Trivy

Trivy is the most practical scanner to integrate. It covers OS packages, language-specific packages (npm, pip, cargo, etc.), and Kubernetes manifests. It has a single binary, fast scan times, and sensible exit code behavior for CI gates.

# .github/workflows/security.yml
name: Container Security Scan

on:
  push:
    branches: [main]
  pull_request:

jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build image
        run: docker build -t myapp:${{ github.sha }} .

      - name: Run Trivy vulnerability scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: myapp:${{ github.sha }}
          format: sarif
          output: trivy-results.sarif
          # Fail on CRITICAL and HIGH severity
          severity: CRITICAL,HIGH
          # Ignore unfixed CVEs to reduce noise
          ignore-unfixed: true
          exit-code: "1"

      - name: Upload scan results to GitHub Security tab
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: trivy-results.sarif

The ignore-unfixed: true flag is worth explaining. Many CVEs in base image packages have no available fix at the time of scan. Blocking builds on unfixed CVEs creates friction with no remediation path, which teams quickly learn to ignore by suppressing all findings. Ignore unfixed; enforce fixed-but-unpatched. That is the set of vulnerabilities you can actually do something about.

Grype as a secondary scanner

Different scanners use different vulnerability databases and matching heuristics. Running both Trivy and Grype catches cases where one misses a finding the other catches. Grype integrates well into pipelines that already use Syft for SBOM generation (covered in the supply chain section):

# Install grype
curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin

# Scan an image, fail on high+ severity with available fixes
grype myapp:latest --fail-on high --only-fixed

For Snyk, the tradeoff is cost versus database quality. Snyk’s vulnerability database is generally considered more current than the open-source alternatives, but it requires a paid account for serious use. At the startup stage, Trivy + Grype covers the important ground for free.

---

Runtime Security Policies

Scanning tells you what is in the image. Runtime policies govern what the process can do while it runs. These are different threat models: scanning addresses known vulnerabilities; runtime policies limit blast radius when a process is compromised.

Seccomp profiles

Seccomp (secure computing mode) filters which system calls a process can make. The default Docker seccomp profile blocks about 44 syscalls that are almost never needed by application code and are commonly used in exploits (e.g., ptrace, personality, keyctl).

For a stricter custom profile, start with a recording run to capture what syscalls your application actually uses, then deny everything else:

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "architectures": ["SCMP_ARCH_X86_64"],
  "syscalls": [
    {
      "names": [
        "accept4", "bind", "brk", "clone", "close", "connect",
        "epoll_create1", "epoll_ctl", "epoll_wait", "execve",
        "exit_group", "fcntl", "fstat", "futex", "getpid",
        "getsockname", "getsockopt", "listen", "mmap", "mprotect",
        "munmap", "nanosleep", "open", "openat", "poll", "read",
        "recvfrom", "recvmsg", "rt_sigaction", "rt_sigprocmask",
        "sendmsg", "sendto", "setsockopt", "socket", "stat",
        "uname", "wait4", "write", "writev"
      ],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

Apply it in Kubernetes via a securityContext annotation or, with Kubernetes 1.19+, natively through the pod spec:

spec:
  securityContext:
    seccompProfile:
      type: Localhost
      localhostProfile: profiles/node-api.json

Generating a minimal allowlist manually is tedious. Tools like docker run --security-opt seccomp=unconfined combined with strace or Falco’s syscall auditing can record what syscalls a real workload makes during test runs, giving you a realistic starting point.

AppArmor profiles

AppArmor operates at a higher level than seccomp: it enforces file access, network access, and capability constraints as named profiles. The Docker runtime ships a default AppArmor profile. A custom profile for a read-heavy API that should never write to disk outside specific paths looks like this:

#include <tunables/global>

profile node-api flags=(attach_disconnected,mediate_deleted) {
  #include <abstractions/base>
  #include <abstractions/nameservice>

  network inet tcp,
  network inet udp,

  # Allow reading application files
  /app/** r,
  /app/node_modules/** r,

  # Allow writing only to tmp
  /tmp/** rw,

  # Deny writes everywhere else
  deny /** w,

  # Allow node binary to execute
  /usr/local/bin/node ix,
}

Load and apply it:

sudo apparmor_parser -r -W /etc/apparmor.d/node-api
# Apply in Docker
docker run --security-opt apparmor=node-api myapp:latest

The operational tradeoff with seccomp and AppArmor is profiling effort. Custom profiles require you to understand your application’s actual behavior and maintain those profiles as the application changes. A dependency upgrade that adds a new syscall or opens a new file path will break a strict profile. Treat profile files as code artifacts: version-control them, test profile changes in staging, and gate production on passing the profiled workload through a test suite.

---

Supply Chain Security

Image signing and SBOM generation address a different threat: the integrity of the artifacts flowing through your pipeline. An unsigned image from your registry could have been substituted by a compromised build system or registry. An SBOM lets you answer “which deployed images are affected by CVE-2024-XXXXX” in minutes rather than hours.

Signing with Cosign and Sigstore

Cosign, part of the Sigstore project, signs container images using OIDC-based keyless signing (no long-lived key files to manage) or traditional key pairs. Keyless signing ties the signature to the OIDC identity of the CI runner, making it possible to verify that an image was built by a specific workflow in a specific repository:

# In GitHub Actions, after pushing the image:
- name: Install Cosign
  uses: sigstore/cosign-installer@v3

- name: Sign image with keyless signing
  env:
    COSIGN_EXPERIMENTAL: "1"
  run: |
    cosign sign --yes \
      ghcr.io/myorg/myapp:${{ github.sha }}

# Verification in a deployment pipeline or admission controller:
- name: Verify image signature
  env:
    COSIGN_EXPERIMENTAL: "1"
  run: |
    cosign verify \
      --certificate-identity-regexp="https://github.com/myorg/myapp/.github/workflows/.*" \
      --certificate-oidc-issuer="https://token.actions.githubusercontent.com" \
      ghcr.io/myorg/myapp:${{ github.sha }}

In a Kubernetes cluster, Policy Controller (also from Sigstore) acts as an admission webhook that rejects pods whose images cannot be verified against your signing policy. This is the production enforcement point: no signature, no admission.

SBOM generation with Syft

An SBOM (Software Bill of Materials) is a structured list of all packages in your image. Generate it as part of the build and attach it to the image as an attestation:

# Generate SBOM with Syft
syft ghcr.io/myorg/myapp:latest -o spdx-json > sbom.spdx.json

# Attach SBOM as a Cosign attestation
cosign attest --predicate sbom.spdx.json \
  --type spdx \
  ghcr.io/myorg/myapp:latest

With SBOMs as attestations, you can query across all running images to find exposure to a specific component:

# Find all images containing a vulnerable version of a package
cosign verify-attestation --type spdx ghcr.io/myorg/myapp:latest \
  | jq '.payload | @base64d | fromjson | .predicate.packages[] | select(.name == "lodash")'

This becomes operationally relevant when a high-severity CVE drops in a widely-used library. With SBOMs attached, you scan your attestations rather than pulling and re-scanning every running image.

---

Secrets Management in Containers

Environment variables are the wrong place for secrets. They are visible in docker inspect, leak into process listings on shared systems, end up in logs when processes print their environment, and get serialized into crash dumps. They also cannot be rotated without redeploying the container.

Mount secrets as files, not environment variables

The recommended pattern is to mount secrets as files at a path your application reads on startup:

# Application reads the secret from a file path
# The path is configurable via an environment variable (not the secret itself)
ENV DB_PASSWORD_FILE=/run/secrets/db_password

// Application code
import { readFileSync } from "fs";

function getSecret(envVar: string): string {
  const filePath = process.env[envVar];
  if (!filePath) {
    throw new Error(`Environment variable ${envVar} not set`);
  }
  try {
    return readFileSync(filePath, "utf8").trim();
  } catch (err) {
    throw new Error(`Failed to read secret from ${filePath}: ${err}`);
  }
}

const dbPassword = getSecret("DB_PASSWORD_FILE");

In Kubernetes, mount secrets as volumes rather than using envFrom:

spec:
  containers:
    - name: api
      image: ghcr.io/myorg/myapp:latest
      volumeMounts:
        - name: db-credentials
          mountPath: /run/secrets
          readOnly: true
  volumes:
    - name: db-credentials
      secret:
        secretName: db-credentials

For external secrets management, External Secrets Operator syncs secrets from AWS Secrets Manager, HashiCorp Vault, or GCP Secret Manager into Kubernetes Secrets automatically. This gives you rotation (the external store holds the source of truth) and auditability (access is logged in the secrets manager).

Never bake secrets into images

This sounds obvious but happens repeatedly. A COPY .env . in a Dockerfile, or a RUN aws configure that caches credentials. Multi-stage builds help here because build-time secrets never reach the final stage. Docker BuildKit’s --mount=type=secret provides a better escape hatch for secrets needed only during build:

# syntax=docker/dockerfile:1.4
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
# Mount the secret only during this RUN step; it is not in the image layer
RUN --mount=type=secret,id=npm_token \
    NPM_TOKEN=$(cat /run/secrets/npm_token) \
    npm ci
COPY . .
RUN npm run build

---

Tradeoffs: Security Controls vs. Developer Velocity

Control	Security Gain	Developer Cost	Priority
Minimal base image	Reduces CVE surface significantly	One-time Dockerfile refactor	High: do first
Non-root user	Limits privilege escalation paths	Minor (file ownership fixes)	High: do first
Read-only filesystem	Blocks persistent writes after compromise	Requires explicit writable volume mounts	High: catches many post-exploit patterns
Vulnerability scanning in CI	Catches known CVEs before deployment	Build time increase (~1-2 min), occasional false positives	High: automate from day one
Seccomp custom profile	Limits exploit syscall paths	Profiling effort; breaks on dependency changes	Medium: use default Docker profile first
AppArmor custom profile	Enforces file and network access at OS level	Significant profiling and maintenance effort	Medium: after seccomp
Image signing (Cosign)	Verifies pipeline provenance	Low (CI step); admission controller setup required	Medium: add when you have a proper registry
SBOM attestation	Enables fast CVE blast-radius analysis	Low (Syft step); storage overhead	Low: add as practice matures
External secrets management	Enables rotation without redeployment	Operational overhead (Vault or external service)	Medium: scales with secret count

The sequencing here is intentional. Minimal images, non-root users, read-only filesystems, and CI scanning give you most of the practical security improvement with the lowest ongoing operational cost. Seccomp and AppArmor custom profiles require maintenance tied to application behavior and are better introduced once the basics are stable. Supply chain controls (signing, SBOMs) are production-grade hygiene that matter more as team size and deployment frequency increase.

---

Production Considerations

Scheduled rescans. Build-time scanning only catches CVEs known at build time. Set up a nightly or weekly rescan of all images in your registry using Trivy or Grype, and alert when new high-severity findings appear in deployed images. Most registries (ECR, GHCR, GCR) offer native scanning; enable it.

Admission control. Vulnerability scanning in CI is a developer-side control. Admission webhooks (OPA/Gatekeeper, Kyverno, or Sigstore Policy Controller) are cluster-side controls that enforce image policies regardless of how a deployment was triggered. A policy that blocks images with critical CVEs or without valid signatures catches images pushed outside the normal CI path.

Image tag hygiene. Mutable tags (latest, main) make it impossible to know exactly what is running in a given environment. Pin all production deployments to immutable digests (ghcr.io/myorg/myapp@sha256:abc123...). Your CI pipeline should output the digest and Kubernetes manifests should reference it.

Falco for runtime anomaly detection. Seccomp and AppArmor enforce static policies. Falco watches for runtime behavioral anomalies: a process spawning a shell, a container writing outside expected paths, unexpected network connections. It complements static policies by covering behavior that is allowed by policy but unusual in practice.

Base image update automation. Set up Dependabot or Renovate to watch your base image versions and open PRs when new versions are released. A monthly manual review is too slow for OS-level CVE timelines.

---

The realistic threat to a containerized production system is not a sophisticated supply chain attack on your image registry. It is a CVE in a transitive dependency that sits unpatched for six months because no one checks, running as root so the blast radius when it is exploited is as wide as possible. The controls in this article address that threat directly. Start with minimal images and non-root users, automate scanning in CI on day one, and add the supply chain and runtime enforcement layers as the operational surface grows.

Container security is not a one-time audit. It is a set of automated gates that run continuously and an SBOM that lets you answer “are we affected?” in minutes rather than hours when the next high-severity CVE drops.