Write-Ahead Logs: How Databases Guarantee Durability and Crash Recovery

You commit a transaction. The database says it succeeded. Your application moves on. Then the server loses power mid-flush, and you restart. The committed data is still there.

That guarantee does not come for free. It is the result of a specific mechanism that every serious storage system uses: the write-ahead log, or WAL.

Understanding WALs matters beyond theoretical curiosity. It explains why databases behave the way they do under load, why fsync matters, what determines recovery time after a crash, and how replication actually works at the storage layer. This article covers the mechanics end-to-end: from the basic invariant to production considerations that bite teams in production.

The Core Invariant

The write-ahead log operates on one rule: before any modification reaches the data files on disk, a record of that modification must be durably written to the log.

That is it. Everything else is implementation detail around enforcing this invariant efficiently.

The reason this rule provides durability is simple. If the system crashes at any point during a write, there are two possible states:

1. The log record was not written. The data file was not modified. Recovery ignores the operation. No inconsistency.
2. The log record was written durably. The data file may or may not have been updated. Recovery replays the log record and ensures the data file is correct.

Without this invariant, a crash during a data file write can leave pages in a partially written state, a “torn write,” with no way to determine what the correct state should be.

Why Sequential Writes Win

Storage performance is not symmetric. On spinning disks, random writes require a physical seek, while sequential writes just advance the write head. The difference is 100x or more in throughput. On SSDs the gap narrows but does not close: sequential writes still reduce write amplification and wear leveling overhead.

The WAL is always written sequentially. New log records are appended to the end. There is never a need to seek back to rewrite a previous position. This makes the WAL extremely fast to write.

Data files are the opposite: they are organized for read performance, which means random access patterns (B-tree pages scattered across the file). Without a WAL, every transaction would require random writes to data files before acknowledging the commit, which is slow and cannot be made durable without careful fsync ordering.

With a WAL, the fast path becomes: append to the log, fsync the log, acknowledge the commit. Data file writes can happen asynchronously in batches, improving throughput significantly.

What fsync Actually Does

fsync(fd) tells the OS to flush all dirty pages for a file from the kernel page cache to the underlying storage device. It blocks until the device acknowledges the write.

This is not the same as write(). A write() call places data in the kernel page cache. The kernel will eventually flush it to disk, but “eventually” is measured in seconds, not microseconds. If the kernel crashes or the machine loses power before the flush, that data is gone.

Calling fsync() after writing each WAL record is what makes the durability guarantee real. Without it, “durable” just means “in RAM with a promise.”

The performance cost of fsync is real. Each fsync involves a round trip to the storage device. On a hard disk, that is roughly 5-10ms. On a SATA SSD, 100-200 microseconds. On NVMe, 20-50 microseconds. Batching multiple log records into a single fsync is a major throughput optimization, which is why databases use group commit: collect all transactions waiting to commit, write all their log records to the WAL in one sequential write, call fsync once, then acknowledge all of them simultaneously.

Checkpointing: Keeping the Log Bounded

The WAL grows with every write. If you never truncated it, you would run out of disk. More practically, you would never want to replay the entire log from the beginning on every restart.

Checkpointing solves this. A checkpoint is a process that:

1. Writes all dirty data file pages to disk (the ones that have WAL records covering them).
2. Records the checkpoint’s position in the WAL.
3. Allows all WAL records before that position to be discarded.

After a checkpoint, crash recovery only needs to replay WAL records from the most recent checkpoint forward, not from the beginning of time. This bounds both disk usage and recovery time.

Checkpointing has its own tradeoffs. During a checkpoint, the database must write a burst of dirty data pages to disk, which competes with normal I/O. Frequent checkpoints keep WAL size small and recovery fast, but increase write amplification. Infrequent checkpoints reduce write amplification but increase recovery time and WAL disk usage. Most databases expose checkpoint frequency and target WAL size as tunable parameters.

How Postgres Implements WAL

Postgres writes WAL records as variable-length entries into 8KB-aligned segment files under $PGDATA/pg_wal/. Each record describes a change at the physical level: which page was modified, what the before and after state looks like.

The WAL writer is a background process that flushes WAL buffers to disk. On commit, Postgres calls fsync (or fdatasync) to guarantee the records are durable before returning to the client. With synchronous_commit = on (the default), every COMMIT waits for WAL to be flushed to disk. With synchronous_commit = off, commits return immediately and WAL is flushed asynchronously, trading a potential loss of the last few seconds of commits for significantly higher throughput.

Postgres checkpoints are controlled by checkpoint_timeout (maximum time between checkpoints, default 5 minutes) and max_wal_size (how much WAL can accumulate before forcing a checkpoint, default 1GB). After a checkpoint completes, WAL segments before the checkpoint LSN can be recycled or archived.

Replication in Postgres is WAL-based. The primary streams WAL records to standbys, which replay them against their own data files. This means a standby is effectively a database that is always replaying the primary’s WAL. Physical replication sends raw WAL bytes; logical replication decodes WAL records into logical change events, which allows more flexibility in what gets replicated and where.

How SQLite Implements WAL

SQLite operates differently: it is an embedded library, not a client-server database. Its WAL mode (enabled with PRAGMA journal_mode=WAL) is designed for the constraint that multiple processes can open the same database file simultaneously.

In default rollback journal mode, SQLite writes modified pages to a journal file before modifying the database file. This is a form of WAL, but the journal is per-transaction and discarded on commit, which limits concurrency: only one writer at a time, and readers block writers.

WAL mode changes the model. Writes go to a WAL file (database.db-wal). Readers access either the database file or the WAL, whichever has the more recent version of a page. Multiple readers and one writer can operate concurrently. The WAL is checkpointed back into the database file periodically.

The critical SQLite WAL detail: the -wal and -shm (shared memory) files are integral parts of the database while WAL mode is active. Copying just the main database file while another process has it open in WAL mode gives you an inconsistent snapshot. Backup must be done through the SQLite backup API or with all connections closed.

How Kafka Uses Logs

Kafka’s architecture is essentially a distributed WAL. Each partition is an append-only log: producers write to the end, consumers read from arbitrary offsets. The offset is the log sequence number.

Kafka does not use a WAL to protect an in-memory data structure the way Postgres does. The log is the data structure. Durability comes from fsync controls (log.flush.interval.messages, log.flush.interval.ms) and from replication: a write is acknowledged only after the configured number of in-sync replicas have written it to their local logs.

Log compaction in Kafka retains the latest record for each key, discarding earlier versions. This is the equivalent of a WAL checkpoint: it bounds log growth while preserving the ability to reconstruct the current state. Compacted topics behave like a database’s current state; non-compacted topics behave like a pure audit log.

A Minimal WAL in TypeScript

Here is a minimal WAL implementation that demonstrates the core mechanics: sequential append, fsync on commit, replay on startup.

import * as fs from "fs";
import * as path from "path";

interface WALRecord {
  lsn: number;
  type: "write" | "commit" | "abort";
  key?: string;
  value?: string;
}

class WriteAheadLog {
  private fd: number;
  private lsn: number = 0;
  private logPath: string;

  constructor(logPath: string) {
    this.logPath = logPath;
    this.fd = fs.openSync(logPath, "a+");
    this.lsn = this.recover();
  }

  // Append a record to the WAL and fsync before returning.
  // This is the durability guarantee: the record is on disk before we proceed.
  append(record: Omit<WALRecord, "lsn">): number {
    const lsn = ++this.lsn;
    const entry = JSON.stringify({ ...record, lsn }) + "\n";
    fs.writeSync(this.fd, entry);
    // fsync ensures the OS flushes the page cache to the device.
    // Without this, the write is only in kernel memory and can be lost on crash.
    fs.fsyncSync(this.fd);
    return lsn;
  }

  close(): void {
    fs.closeSync(this.fd);
  }

  // Read all records from the log file and return them in order.
  // Called during startup to replay any uncommitted transactions.
  readAll(): WALRecord[] {
    const content = fs.readFileSync(this.logPath, "utf8");
    return content
      .split("\n")
      .filter((line) => line.trim().length > 0)
      .map((line) => JSON.parse(line) as WALRecord);
  }

  // Replay the log and return the last committed LSN.
  // Any writes without a corresponding commit record are discarded.
  private recover(): number {
    if (!fs.existsSync(this.logPath)) return 0;

    const records = this.readAll();
    if (records.length === 0) return 0;

    const committedLSNs = new Set(
      records.filter((r) => r.type === "commit").map((r) => r.lsn)
    );

    // In a real implementation, we would apply writes to the data store here,
    // but only for transactions whose commit record is present in the log.
    for (const record of records) {
      if (record.type === "write" && committedLSNs.has(record.lsn)) {
        // Apply record.key / record.value to the in-memory store.
      }
    }

    return Math.max(...records.map((r) => r.lsn));
  }
}

// Usage: a simple key-value store backed by a WAL.
class DurableKVStore {
  private wal: WriteAheadLog;
  private store: Map<string, string> = new Map();

  constructor(walPath: string) {
    this.wal = new WriteAheadLog(walPath);
    // On startup, replay the WAL to rebuild in-memory state.
    this.replayFromWAL();
  }

  set(key: string, value: string): void {
    // Write the record to the WAL first, then update in-memory state.
    // If we crash after the WAL write but before updating the map,
    // the next startup will replay the WAL and recover the write.
    const lsn = this.wal.append({ type: "write", key, value });
    this.wal.append({ type: "commit" });
    this.store.set(key, value);
  }

  get(key: string): string | undefined {
    return this.store.get(key);
  }

  private replayFromWAL(): void {
    const records = this.wal.readAll();
    const committed = new Set(
      records.filter((r) => r.type === "commit").map((r) => r.lsn - 1)
    );
    for (const record of records) {
      if (record.type === "write" && committed.has(record.lsn) && record.key) {
        this.store.set(record.key, record.value ?? "");
      }
    }
  }
}

This implementation skips transaction batching, log compaction, and checkpointing, but captures the essential contract: write to the log with fsync before touching state, replay on startup to handle crashes.

WAL vs. In-Place Updates: The Tradeoffs

Factor	WAL	In-Place Update
Write throughput	High (sequential appends, group commit)	Lower (random writes, per-write fsync)
Read performance	Requires consulting log for recent writes	Direct page access
Crash recovery	Replay from last checkpoint	Requires external journal or snapshot
Disk space	Growing log requires periodic checkpointing	Stable (overwrites existing pages)
Replication	Natural: stream the log	Complex: track and ship page-level diffs
Implementation complexity	Higher	Lower
MVCC support	Natural fit (old versions can be in log or heap)	Requires undo log or version chain

In-place updates without a journal exist in older or simpler storage engines. They are acceptable when durability is explicitly not a requirement (caches, ephemeral state) or when the storage layer provides durability through other means (EBS snapshots, RAID). For any database that guarantees ACID transactions, WAL or an equivalent journal is not optional.

Crash Recovery Sequence

When a database starts after an unclean shutdown, it follows a deterministic recovery sequence. Postgres uses ARIES (Algorithm for Recovery and Isolation Exploiting Semantics), but the high-level steps are consistent across systems:

Analysis pass: scan the WAL from the last checkpoint forward to determine which transactions were in progress at the time of the crash, and which data pages were dirty.

Redo pass: replay all WAL records from the oldest dirty page LSN forward, unconditionally. This brings data files to the state they would have been in if every write had been flushed before the crash. This may re-apply operations that were already in the data files, but ARIES handles this through LSN comparisons on pages.

Undo pass: for any transaction that did not reach a COMMIT record in the WAL, reverse its changes using the undo information embedded in WAL records. This removes partial writes from transactions that never committed.

After the undo pass, the database is in a consistent state: all committed transactions are present, all uncommitted transactions are rolled back.

Recovery time scales with the amount of WAL between the last checkpoint and the crash point. A database that had not checkpointed for 20 minutes before crashing will spend roughly that amount of time on recovery. Aggressive checkpointing reduces recovery time at the cost of more frequent I/O during normal operation.

Production Considerations

WAL size and disk space: WAL accumulates between checkpoints and when replication lag grows. A standby that falls behind means the primary must retain WAL records the standby has not yet consumed. In Postgres, wal_keep_size and replication slots control this. A replication slot with a lagging standby can cause unbounded WAL accumulation and fill the disk. Monitor replication lag and WAL directory size aggressively.

Disk I/O patterns: WAL writes are sequential and predictable, but checkpoint writes are bursty and random. On systems with high write throughput, checkpoint I/O can spike latency for normal queries. Spread the load with checkpoint_completion_target (Postgres default 0.9), which spreads checkpoint writes across 90% of the checkpoint interval rather than flushing everything at once.

Separate WAL disk: placing WAL files on a dedicated disk (or dedicated NVMe) isolates the sequential WAL I/O from the random data file I/O. This is the single highest-impact hardware change for write-heavy workloads. The WAL disk does not need large capacity, it needs low write latency and high write IOPS.

Synchronous vs. asynchronous commit: turning off synchronous commit (synchronous_commit = off in Postgres) allows up to wal_writer_delay (default 200ms) of committed transactions to be lost on crash. For workloads that tolerate this (logging pipelines, analytics event ingestion), the throughput gain is significant. For financial transactions or any data the user expects to be durable, keep it on.

WAL compression: Postgres 15+ supports WAL compression (wal_compression). On workloads with compressible data patterns, this reduces WAL volume significantly, which reduces replication bandwidth and checkpoint I/O. There is a small CPU cost on the write path.

Archive and point-in-time recovery (PITR): archiving WAL segments to object storage enables PITR: restoring the database to any point in time by replaying the base backup through archived WAL segments. This is the foundation of most database backup strategies for production systems. Test restoration regularly. An archive that has never been restored is a hypothesis, not a backup.

The Invariant as Design Principle

The WAL is not a database-specific curiosity. The same pattern appears anywhere you need durability: event sourcing systems use an immutable event log as the source of truth; distributed consensus algorithms like Raft require log entries to be durably written before they can be applied; Kafka’s partition log gives consumers the ability to replay from any offset.

The underlying principle is the same in all cases: durable, sequential writes are cheap and safe. In-place mutations are fast but fragile. When correctness is required, write your intent to the log before you act on it.

Once you understand WAL mechanics, a lot of database behavior that seems arbitrary starts to make sense: why checkpoints cause I/O spikes, why replication lag affects disk usage, why fsync = off is a footgun, why recovery time correlates with checkpoint interval. These are not implementation quirks but direct consequences of the core invariant.