How Package Managers Work: Dependency Resolution, Lockfiles, and the Architecture Behind npm, pnpm, and Yarn

Every JavaScript developer runs npm install dozens of times a day. Most treat it as a black box: you ask for react@^18 and the folder appears. When it breaks, though, the failure modes are subtle. Two developers on the same team get different transitive versions. A fresh CI build produces a subtly different dependency tree. A package works locally but fails in production because a phantom dependency got hoisted differently.

Understanding what actually happens inside a package manager turns these mysteries into diagnosable problems. This article covers the full stack: constraint satisfaction during resolution, how lockfiles encode the resolved graph, the flat versus nested node_modules debate, pnpm’s content-addressable store with hard links, and the HTTP registry protocol that ties it together.

The Resolution Problem

A package manifest (package.json) expresses constraints, not versions. "react": "^18.2.0" means “any 18.x at 18.2.0 or above.” The package manager’s job is to find a concrete version for every constraint in the entire dependency graph, including the transitive ones you never wrote yourself.

This is a form of constraint satisfaction. The resolver traverses the dependency graph breadth-first, collecting all version constraints for each package name, and then selects a version that satisfies all of them simultaneously.

type VersionRange = string; // semver range string like "^18.2.0"
type ConcreteVersion = string; // "18.3.1"

interface PackageManifest {
  name: string;
  version: ConcreteVersion;
  dependencies: Record<string, VersionRange>;
}

interface ResolutionNode {
  name: string;
  version: ConcreteVersion;
  resolvedDependencies: Map<string, ResolutionNode>;
}

// Simplified resolver: collect all version ranges for a given package name,
// then find the highest version that satisfies all of them.
function resolveConstraints(
  ranges: VersionRange[],
  availableVersions: ConcreteVersion[]
): ConcreteVersion | null {
  // semver.satisfies from the "semver" package does the actual matching
  const candidates = availableVersions
    .filter((v) => ranges.every((r) => semverSatisfies(v, r)))
    .sort(semverCompare)
    .reverse();

  return candidates[0] ?? null;
}

The hard part is that constraints can conflict. Package A requires lodash@^4.17.0 and package B requires lodash@^3.10.0. Those ranges share no common version. The package manager must then decide: install two copies (one per dependent) or report an error. npm and Yarn install two copies silently. This is the root cause of the bundle size and compatibility issues you see with older dependency trees.

Version Range Semantics

Semver ranges are more expressive than most people realize:

Range Syntax	Meaning
^18.2.0	>=18.2.0 <19.0.0 (caret: compatible with)
~18.2.0	>=18.2.0 <18.3.0 (tilde: patch-level)
18.2.0	Exact version
>=18.2.0 <19	Explicit range
*	Any version
18.x	Any patch within 18

The caret (^) deserves special attention. For packages at 1.0.0 or above, it allows minor and patch updates. For packages below 1.0.0 (like ^0.8.0), it only allows patch updates because pre-1.0 packages treat minor bumps as breaking. This trips people up constantly.

Hoisting and the Flat node_modules Layout

npm v3 and later, along with Yarn Classic, flatten the dependency tree into a single node_modules directory at the project root. Before v3, every package carried its own nested node_modules inside it, which led to deeply nested paths that exceeded Windows’s MAX_PATH limit and duplicated packages across the tree even when a single version would have satisfied everyone.

Hoisting works by promoting packages as high in the tree as possible. If both your app and five of your dependencies all want lodash@4.17.21, hoisting places one copy at node_modules/lodash and everyone resolves to it. Only when two dependencies need incompatible versions does a second copy get nested inside the requiring package’s own node_modules.

node_modules/
  lodash/              <- hoisted, version 4.17.21
  express/
    node_modules/
      lodash/          <- nested copy, version 3.10.1 (incompatible constraint)
  react/
  webpack/

The algorithm that determines which version gets the top-level slot is non-deterministic in older npm versions. The first package encountered during resolution wins the top-level slot, and install order depends on the order packages appear in package.json. This is why two developers could get different trees from the same manifest.

Phantom Dependencies

Hoisting creates a serious correctness problem: phantom dependencies. Because everything at any nesting level can require() from the top-level node_modules, your code can import a package you never declared in your own package.json. The package was hoisted there by some other dependency.

// Your package.json has no dependency on "chalk"
// but chalk was hoisted because "jest" depends on it

import chalk from "chalk"; // works today, breaks silently when jest stops using chalk

This is not a hypothetical. It happens in real projects all the time. You update a dependency, its transitive chalk dependency bumps from v4 to v5 (which dropped CommonJS), and suddenly your code breaks because the version under your feet changed without you touching anything.

Lockfiles: Encoding the Resolved Graph

A lockfile is the output of resolution frozen to disk. It maps every package name and version range to a specific concrete version, a download URL, and a content hash. Future installs skip resolution entirely and read directly from the lockfile.

Here is a fragment from package-lock.json v3 (npm):

{
  "name": "my-app",
  "lockfileVersion": 3,
  "packages": {
    "node_modules/react": {
      "version": "18.3.1",
      "resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
      "integrity": "sha512-wS+hAgJShR0KhEvPJArfuPVN1+Hz1t0Y6n5jLASyDNYFVajNh3HEFp7UxOGFMpVeUP4qd0f9uf+e8uIhEPVQ==",
      "dependencies": {
        "loose-envify": "^1.1.0"
      }
    },
    "node_modules/loose-envify": {
      "version": "1.4.0",
      "resolved": "https://registry.npmjs.org/loose-envify/-/loose-envify-1.4.0.tgz",
      "integrity": "sha512-lyuxPGr/Wfhrlem2CL/UcnUc1zcqKAImBDzukY7Y5F/yQiNdko6+DVGd+OEL6orn6nE8Y2hwkqGHPC7/XjIQ=="
    }
  }
}

The integrity field is a subresource integrity hash (SHA-512, base64-encoded). When npm fetches the tarball, it verifies the hash before unpacking. This guarantees that even if the registry gets compromised or a package is mutated after publish, you will never install code different from what was locked.

Yarn uses yarn.lock with its own format. pnpm uses pnpm-lock.yaml. The semantics are the same: resolved version plus integrity hash for every node in the graph.

Critical rule: commit your lockfile to version control. Every time you deploy from a CI environment without a lockfile, you are running resolution again, and you might get different results. npm ci (versus npm install) enforces this by refusing to run if the lockfile is absent or out of sync with package.json.

// What "npm ci" enforces conceptually
function ciInstall(manifest: PackageManifest, lockfile: Lockfile): void {
  validateLockfileMatchesManifest(manifest, lockfile); // throws if out of sync
  deleteExistingNodeModules();
  installExactlyFromLockfile(lockfile); // no resolution, no network guessing
}

pnpm: Content-Addressable Store and Hard Links

pnpm takes a fundamentally different approach to storing packages. Instead of copying package files into each project’s node_modules, pnpm maintains a single content-addressable store on the filesystem (default: ~/.pnpm-store). Each file in the store is keyed by its SHA-512 hash. When two packages across all your projects share the same file content, only one copy exists on disk.

~/.pnpm-store/
  v3/
    files/
      00/
        3a7f...  <- file content addressed by hash
      1b/
        c4e2...
      ...

Inside your project’s node_modules, pnpm creates hard links to the store. A hard link is a filesystem-level alias: it points to the same inode (and therefore the same bytes) as the original file, but appears as a normal file in the directory listing. The result is zero extra disk space per project for packages you have already installed elsewhere.

// Conceptual model of pnpm's install process
async function pnpmInstall(packages: ResolvedPackage[]): Promise<void> {
  for (const pkg of packages) {
    for (const file of pkg.files) {
      const hash = sha512(file.content);
      const storePath = path.join(STORE_DIR, "files", hash.slice(0, 2), hash.slice(2));

      if (!existsSync(storePath)) {
        await writeFile(storePath, file.content); // write once to store
      }

      const linkPath = path.join("node_modules/.pnpm", pkg.id, "node_modules", pkg.name, file.relativePath);
      await fs.link(storePath, linkPath); // hard link, no copy
    }
  }
}

Beyond disk savings, the store enables near-instant installs for packages you have seen before. If all the hashes already exist in the store, the install is just creating hard links, which is a metadata operation and takes milliseconds regardless of package size.

pnpm’s Strict node_modules Layout

pnpm does not hoist packages to the top-level node_modules. Instead, it uses a virtual store at node_modules/.pnpm where every package version lives in an isolated directory. The top-level node_modules only contains symlinks to the packages you explicitly declared in your own package.json.

node_modules/
  react -> .pnpm/react@18.3.1/node_modules/react   (symlink)
  .pnpm/
    react@18.3.1/
      node_modules/
        react/           <- hard links to store
        loose-envify/    <- react's own dependency, also hard linked
    express@4.18.2/
      node_modules/
        express/
        body-parser/     <- express's dependency

This layout eliminates phantom dependencies at the filesystem level. If you try to import a package you did not declare, Node.js’s module resolution will walk up the directory tree and fail to find it in your top-level node_modules because it is not symlinked there. You get an error immediately at install time or first require, not a silent breakage weeks later when someone bumps a transitive dependency.

The Registry Protocol

When the resolver needs a version it does not have cached, it fetches metadata and tarballs from a registry. The npm registry exposes a simple HTTP API.

Fetching all metadata for a package:

GET https://registry.npmjs.org/react

This returns a JSON document (called a packument) containing metadata for every published version: dependency lists, dist-tags (like latest), tarball URLs, and integrity hashes. The packument for popular packages like lodash is large: the full document is several megabytes.

Fetching metadata for one specific version:

GET https://registry.npmjs.org/react/18.3.1

Fetching the tarball:

GET https://registry.npmjs.org/react/-/react-18.3.1.tgz

Modern package managers cache packuments aggressively. npm stores them in ~/.npm/_cacache. pnpm stores them in its content-addressable store. This is why the second install of the same set of packages is much faster than the first: the packuments and tarballs are already local.

Private registries (Verdaccio, Artifactory, GitHub Packages) implement the same HTTP interface, which is how npm install --registry https://npm.your-company.com works transparently.

Tradeoffs: npm vs pnpm vs Yarn

Dimension	npm	pnpm	Yarn Berry
node_modules layout	Flat (hoisted)	Strict (symlinks + .pnpm)	Plug’n’Play (no node_modules)
Phantom dependencies	Possible (hoisting)	Blocked by layout	Blocked by PnP
Disk usage	Per-project copy	Shared store (hard links)	Zip archives in cache
Install speed	Baseline	Faster on repeated installs	Fast, but PnP setup cost
Compatibility	Universal	High (most tools work)	Lower (PnP breaks some tools)
Monorepo support	Workspaces (basic)	Workspaces (strong)	Workspaces (strong)
Lockfile format	package-lock.json	pnpm-lock.yaml	yarn.lock
Strictness	Permissive	Strict by default	Configurable

Yarn Berry’s Plug’n’Play mode is the most radical departure. It eliminates node_modules entirely, storing packages as zip archives and using a generated .pnp.cjs file as the resolver. This removes hoisting-related bugs completely, but many tools (webpack loaders, Jest’s transform pipeline, editors) require additional configuration because they assume node_modules exists on disk.

Production Guidance

Use pnpm for new projects. The strict layout catches phantom dependency bugs early rather than letting them surface in production. The shared store pays for itself immediately on any machine running more than one JavaScript project. The tradeoff is that a small number of legacy packages assume hoisting and break under pnpm’s strict mode, but the fix is almost always a one-line .npmrc setting or a peer dependency declaration.

Commit your lockfile without exception. Whether it is package-lock.json, pnpm-lock.yaml, or yarn.lock, the lockfile is what makes your dependency tree reproducible. CI should always run npm ci, pnpm install --frozen-lockfile, or yarn install --immutable.

Audit your phantom dependencies before migrating to pnpm. Run pnpm install on an existing npm project and watch for module-not-found errors. Each one is a phantom dependency that was silently working. Fix them by adding the actual dependency to your package.json.

Pin major versions for infrastructure packages. For packages like TypeScript, ESLint, and your bundler, use exact versions or tilde ranges (~5.4.0) rather than caret. These packages have a history of subtle breaking changes in minor bumps. The cost of occasionally running npm update manually is much lower than debugging a broken build after a CI cache bust.

Watch your lockfile diffs in code review. A lockfile change that adds 40 new transitive dependencies for a feature that should have added two is a signal worth investigating. Automated tools like lockfile-lint and Renovate’s PR descriptions help, but human review catches the cases tooling misses.

Closing

Package managers are dependency constraint solvers layered on top of a content-addressed file store and an HTTP registry protocol. The resolution step is where version conflicts surface. The lockfile is what turns a probabilistic system into a deterministic one. The node_modules layout determines whether phantom dependencies are possible at all.

Most production incidents attributed to “a bad deploy” are actually lockfile drift, phantom dependency breakage after a transitive update, or a resolution difference between a developer’s machine and CI. Understanding the mechanics means you can reproduce the failure and fix the root cause rather than just retrying the deploy.