System Design ·

Event Schema Versioning in Practice: Evolution Strategies, Registry Patterns, and Breaking Change Management for Production Event Systems

How to evolve event schemas in production event-driven architectures without breaking consumers. Covers compatibility modes, schema registry patterns, TypeScript implementations with Zod, upcasting, dual-write migration, and CI/CD enforcement.

Event Schema Versioning in Practice: Evolution Strategies, Registry Patterns, and Breaking Change Management for Production Event Systems

You renamed a field. You deployed the producer. The consumer was deployed last week and still expects the old name. Now you have a silent deserialization failure at 2 AM and a pile of events in a dead-letter queue that nobody owns.

This is not a hypothetical. Schema drift in event-driven systems is one of the most common sources of production incidents that teams do not fully understand until they are already on fire. The core problem is that events are not API calls. With a REST endpoint you can version the URL, coordinate a cutover, and deprecate cleanly. Events are asynchronous, durable, and consumed by multiple services that deploy on different schedules. The producer and consumer are decoupled by design, which means the schema contract between them has no enforcement point by default.

This article covers the full surface: compatibility modes, schema registry patterns, TypeScript implementations with Zod, migration strategies for existing event stores, and how to wire schema validation into CI.

Why Schema Drift Happens in Practice

Consider a typical event-driven order system. The OrderPlaced event starts as:

interface OrderPlaced {
  orderId: string;
  customerId: string;
  totalAmount: number;
  items: Array<{ productId: string; quantity: number; price: number }>;
  createdAt: string;
}

Six months later, the team needs to support multi-currency. Someone adds currency: string and renames totalAmount to amount. They update the producer, run the integration tests (which were also updated), and ship. The consumers that were already deployed and running in production never got the memo. The result: every event emitted after the deploy is silently parsed as { amount: undefined } in consumers that still expect totalAmount.

This is a breaking change. The rename is not backward-compatible. Consumers written against the old schema cannot process the new event without modification.

The failure is predictable and preventable, but only if you have a shared understanding of what “compatible” means and a mechanism to enforce it.

Compatibility Modes

Before picking a migration strategy, you need a vocabulary for what kinds of changes are safe.

Backward compatibility means new schema versions can be read by consumers written against old schema versions. Adding an optional field is backward-compatible. Removing a required field, renaming a field, or changing a field’s type is not.

Forward compatibility means old schema versions can be read by consumers written against new schema versions. This is useful when producers lag behind consumers. Removing a field is forward-compatible if the consumer handles its absence gracefully. Adding a required field is not forward-compatible because older producers will not emit it.

Full compatibility means both backward and forward: new and old versions can interoperate in either direction. This is the most restrictive mode and essentially limits you to adding optional fields and never removing anything.

In practice, most teams should target backward compatibility as the default. It lets consumers lag behind producers without breaking. Full compatibility is worth enforcing in high-stakes shared schemas. Forward-only scenarios are rare outside of embedded device contexts.

Here is a concrete TypeScript mapping of which changes fall into which category:

type CompatibilityMode = "backward" | "forward" | "full" | "none";

interface SchemaChange {
  description: string;
  backward: boolean; // old consumers can read new events
  forward: boolean;  // new consumers can read old events
}

const changeCompatibility: SchemaChange[] = [
  { description: "Add optional field",           backward: true,  forward: true  },
  { description: "Add required field",           backward: false, forward: true  },
  { description: "Remove optional field",        backward: true,  forward: false },
  { description: "Remove required field",        backward: false, forward: false },
  { description: "Rename field",                 backward: false, forward: false },
  { description: "Widen field type (num→string)",backward: false, forward: false },
  { description: "Narrow field type (any→enum)", backward: false, forward: true  },
  { description: "Change field from required→optional", backward: true, forward: false },
];

This table is worth printing and hanging near whoever reviews schema PRs.

Schema Registry Patterns

Once you have a compatibility vocabulary, you need an enforcement mechanism. Three patterns are in common use.

Centralized Schema Registry

A centralized registry (Confluent Schema Registry is the canonical example, but you can build a lightweight version) stores versioned schemas and validates compatibility on registration. Producers register a new schema version before publishing. Consumers fetch the schema by ID embedded in the event envelope.

The event wire format looks like:

interface EventEnvelope<T = unknown> {
  schemaId: string;    // e.g., "order.placed@v3"
  schemaVersion: number;
  eventId: string;
  occurredAt: string;
  payload: T;
}

Consumers resolve the schema from the registry at startup or on first receipt, then validate against it:

import { z } from "zod";

class SchemaRegistry {
  private schemas = new Map<string, z.ZodTypeAny>();

  register(schemaId: string, schema: z.ZodTypeAny): void {
    this.schemas.set(schemaId, schema);
  }

  resolve(schemaId: string): z.ZodTypeAny {
    const schema = this.schemas.get(schemaId);
    if (!schema) {
      throw new Error(`Unknown schema: ${schemaId}`);
    }
    return schema;
  }

  validate(schemaId: string, payload: unknown): unknown {
    return this.resolve(schemaId).parse(payload);
  }
}

The main advantage of a centralized registry is that compatibility can be checked at registration time, before the new producer version ever goes to production. The downside is operational: it is another service to run, and consumers have a runtime dependency on its availability.

Embedded Schemas (Schema-per-Event)

A lighter-weight alternative is to embed the schema definition alongside the event type in a shared package. Each event type carries its own Zod schema, and both producer and consumer import from the same package.

// packages/events/src/order-placed.ts
import { z } from "zod";

export const OrderPlacedV2Schema = z.object({
  orderId: z.string().uuid(),
  customerId: z.string().uuid(),
  currency: z.string().length(3),
  amount: z.number().positive(),
  items: z.array(z.object({
    productId: z.string(),
    quantity: z.number().int().positive(),
    price: z.number().nonnegative(),
  })),
  createdAt: z.string().datetime(),
});

export type OrderPlacedV2 = z.infer<typeof OrderPlacedV2Schema>;

// Keep previous versions for consumers that have not yet migrated
export const OrderPlacedV1Schema = z.object({
  orderId: z.string().uuid(),
  customerId: z.string().uuid(),
  totalAmount: z.number().positive(),
  items: z.array(z.object({
    productId: z.string(),
    quantity: z.number().int().positive(),
    price: z.number().nonnegative(),
  })),
  createdAt: z.string().datetime(),
});

export type OrderPlacedV1 = z.infer<typeof OrderPlacedV1Schema>;

The shared package becomes the contract. Bump the package version on any schema change. Consumers pin the package version they support. This avoids a runtime registry dependency, but version drift across services is still possible because not all services upgrade the package at the same time.

Contract Testing

A third option is to encode schema compatibility as automated tests run in CI. Consumer-driven contract testing (Pact is the canonical tool for HTTP; a similar pattern works for events) has each consumer publish a “contract” describing the event structure it requires. The producer’s CI pipeline runs all consumer contracts against the new schema and fails if any contract breaks.

For a TypeScript event system, a lightweight version of this pattern works without a dedicated contract testing tool:

// contracts/order-placed-consumer-a.contract.ts
import { OrderPlacedV2Schema } from "@your-org/events";

// This file is the contract: if OrderPlacedV2Schema changes in a way
// that makes this parse fail, CI catches it before the producer ships.
const exampleEvent = {
  orderId: "018e7a1d-3a72-7000-b6f5-123456789abc",
  customerId: "018e7a1d-3a72-7000-b6f5-987654321def",
  currency: "USD",
  amount: 99.95,
  items: [{ productId: "prod-1", quantity: 2, price: 49.97 }],
  createdAt: "2026-05-03T12:00:00.000Z",
};

// If this throws, the CI step fails
OrderPlacedV2Schema.parse(exampleEvent);

This is not a full Pact replacement, but it catches the most common class of breakage without operational overhead.

Versioned Producers and Consumers in TypeScript

The core implementation pattern for multi-version support is an event dispatcher that emits a discriminated union and a consumer that handles each version explicitly.

// Producer: always emits the latest version
import { randomUUID } from "crypto";
import type { OrderPlacedV2 } from "@your-org/events";
import { OrderPlacedV2Schema } from "@your-org/events";

function emitOrderPlaced(data: OrderPlacedV2): void {
  const event: EventEnvelope<OrderPlacedV2> = {
    schemaId: "order.placed",
    schemaVersion: 2,
    eventId: randomUUID(),
    occurredAt: new Date().toISOString(),
    payload: OrderPlacedV2Schema.parse(data), // validate before emit
  };
  // publish to broker...
}

The consumer handles both versions during the transition window:

import { OrderPlacedV1Schema, OrderPlacedV2Schema } from "@your-org/events";
import type { OrderPlacedV1, OrderPlacedV2 } from "@your-org/events";

type VersionedOrderPlaced =
  | { version: 1; payload: OrderPlacedV1 }
  | { version: 2; payload: OrderPlacedV2 };

function parseOrderPlaced(envelope: EventEnvelope): VersionedOrderPlaced {
  switch (envelope.schemaVersion) {
    case 1:
      return { version: 1, payload: OrderPlacedV1Schema.parse(envelope.payload) };
    case 2:
      return { version: 2, payload: OrderPlacedV2Schema.parse(envelope.payload) };
    default:
      throw new Error(`Unsupported schema version: ${envelope.schemaVersion}`);
  }
}

function handleOrderPlaced(envelope: EventEnvelope): void {
  const event = parseOrderPlaced(envelope);

  // Normalize to latest before processing
  const normalized: OrderPlacedV2 = event.version === 1
    ? upcastV1ToV2(event.payload)
    : event.payload;

  processOrder(normalized);
}

The upcastV1ToV2 function is the upcaster: a pure function that transforms an old event shape into a new one.

Migration Strategies: Upcasting, Lazy Migration, Dual-Write

When you already have an event store with thousands or millions of events in an older schema, you have three options.

Upcasting transforms old events to the new schema at read time. The event store is not modified. Every consumer applies an upcasting chain before processing. This keeps the store immutable (which is correct for event sourcing) at the cost of upcasting overhead on every read.

function upcastV1ToV2(v1: OrderPlacedV1): OrderPlacedV2 {
  return {
    orderId: v1.orderId,
    customerId: v1.customerId,
    currency: "USD",          // default for legacy events
    amount: v1.totalAmount,   // rename: totalAmount → amount
    items: v1.items,
    createdAt: v1.createdAt,
  };
}

The currency: "USD" default here is a business decision that should be documented alongside the upcaster, not buried in code.

Lazy migration writes the new schema version back to the store when an old event is read and processed. This amortizes the migration cost over time without a big-bang rewrite. The tradeoff is a more complex read path and a long tail of old-format events.

Dual-write is appropriate when switching event schemas for forward-looking events. The producer writes both old and new schema versions simultaneously during a transition window. Once all consumers have been updated to handle the new version, the old write is removed. This is the safest approach for live systems with many consumers, but it doubles write load during the window.

For most systems, upcasting at the consumer is the right default because it preserves event store immutability and can be deployed incrementally.

Production Tradeoffs

ApproachCompatibility EnforcementOperational CostBreaking Change RecoveryRecommended For
Centralized registryAt registration timeMedium (registry service)Blocked before productionHigh-scale systems, many teams
Embedded schemas (shared pkg)At package upgradeLowCaught at compile timeSingle-team or mono-repo setups
Contract testing in CIAt producer CILowCaught before mergeAny system; complements registry
No registry (hope)NoneNoneIncident at runtimeNever
Upcasting at consumerAt read timeLowLocalized to consumerEvent-sourced systems
Dual-write transitionDuring windowMedium (double writes)Gradual, reversibleLive systems with many consumers

CI/CD Integration for Schema Validation

Schema validation belongs in the pipeline before events reach the broker. A minimal CI check for a TypeScript monorepo:

// scripts/validate-schemas.ts
import { readdirSync, readFileSync } from "fs";
import { join } from "path";
import Ajv from "ajv";
import { zodToJsonSchema } from "zod-to-json-schema";
import * as EventSchemas from "../packages/events/src";

const ajv = new Ajv({ strict: true });

// For each exported schema, verify it compiles to valid JSON Schema
for (const [name, schema] of Object.entries(EventSchemas)) {
  if (!name.endsWith("Schema")) continue;

  try {
    const jsonSchema = zodToJsonSchema(schema as any, name);
    ajv.compile(jsonSchema);
    console.log(`  ${name}: valid`);
  } catch (err) {
    console.error(`  ${name}: INVALID — ${(err as Error).message}`);
    process.exit(1);
  }
}

A more complete pipeline step runs backward-compatibility checks by comparing the new schema against the previous version stored in the registry or in git. If any breaking change is detected, the step fails. This catches renames, type changes, and removed required fields before they reach the broker.

Schema Governance and Ownership

Technical enforcement only works if the organizational side is also in place. A few patterns that actually hold up in production:

Schema changes require a PR that touches both the schema definition and any upcasters or contract files. This forces a conversation before code merges, not after an incident.

Every event type should have a declared owner (a team, not a person). When the event changes, the owner is responsible for coordinating consumer updates and managing the transition window. Events without declared owners are the ones that cause 2 AM pages.

Deprecation should be explicit and time-boxed. A field marked deprecated should carry a sunset date in the schema comment or a structured annotation. Consumers have until the sunset date to migrate. After that, the field is removed. Without a deadline, deprecated fields live forever.

export const OrderPlacedV3Schema = z.object({
  orderId: z.string().uuid(),
  customerId: z.string().uuid(),
  currency: z.string().length(3),
  amount: z.number().positive(),
  // totalAmount deprecated: remove after 2026-08-01; use `amount` instead
  totalAmount: z.number().positive().optional(),
  items: z.array(z.object({
    productId: z.string(),
    quantity: z.number().int().positive(),
    price: z.number().nonnegative(),
  })),
  createdAt: z.string().datetime(),
});

Running a sunset checker in CI that reads structured deprecation annotations and fails if a sunset date has passed is a low-effort way to prevent the “deprecated but never removed” accumulation that makes schemas unreadable over time.

Production Considerations

The most common failure mode is not a dramatic breaking change but slow drift: optional fields added carelessly, undocumented defaults, no upcasters for old events in the store. The system works until someone replays historical events or a consumer that processes the full event log comes online.

The registry pattern you choose matters less than having a pattern at all. A shared package with strict semantic versioning and contract tests in CI will catch most breaking changes for a small-to-medium team. A centralized registry adds value when multiple teams publish to shared topics and cannot coordinate deploys directly.

Keep upcasters colocated with the schema definition, not scattered across consumer codebases. When you need to understand what a v1 event looked like and how it maps to v4, the upcasting chain should be readable in one place. Document the business decisions embedded in upcasters (like the currency: "USD" default) as explicitly as you document the code.

Schema governance is a coordination problem. The tools help, but the work is making sure that anyone who wants to change an event schema knows the cost, the process, and who they need to talk to before merging.

More in System Design

How Amazon Aurora Works Internally: The Log-Is-the-Database Architecture, Quorum Writes, and Storage-Compute Separation Behind Cloud-Native SQL
System Design ·

How Amazon Aurora Works Internally: The Log-Is-the-Database Architecture, Quorum Writes, and Storage-Compute Separation Behind Cloud-Native SQL

A deep dive into Aurora's storage-compute separation, the log-is-the-database design that pushes redo log processing to storage nodes, quorum-based replication across 6 copies in 3 AZs, protection group architecture, fast cloning, Serverless v2 scaling mechanics, and honest tradeoffs versus RDS, self-managed PostgreSQL, CockroachDB, and Cloud Spanner.

How CockroachDB Works Internally: Distributed SQL, Raft Consensus Per Range, and the Architecture Behind Serializable Transactions at Global Scale
System Design ·

How CockroachDB Works Internally: Distributed SQL, Raft Consensus Per Range, and the Architecture Behind Serializable Transactions at Global Scale

A deep dive into CockroachDB's internals: the 512MB range-based data model with automatic splitting, per-range Raft replication, MVCC timestamp ordering with hybrid logical clocks, DistSQL physical planning, serializable transactions with timestamp refreshes, closed timestamps for follower reads, online schema changes using multi-version schema descriptors, and production considerations for hotspots and write amplification.

How Container Runtimes Work Internally: Namespaces, Cgroups, and the OCI Stack From Docker Run to Process Isolation
System Design ·

How Container Runtimes Work Internally: Namespaces, Cgroups, and the OCI Stack From Docker Run to Process Isolation

Containers are not virtual machines and they are not magic. They are a thin composition of Linux kernel primitives: namespaces, cgroups, and a layered filesystem. This article traces exactly what happens between docker run and a running process, covering the OCI spec, containerd, runc, overlay filesystems, and container networking at the kernel level.

How YugabyteDB Works Internally: DocDB Storage, Tablet Splitting, and the Dual API Architecture That Scales PostgreSQL Horizontally
System Design ·

How YugabyteDB Works Internally: DocDB Storage, Tablet Splitting, and the Dual API Architecture That Scales PostgreSQL Horizontally

A deep dive into YugabyteDB internals covering the DocDB storage engine built on RocksDB LSM trees, per-tablet Raft consensus, YSQL and YCQL dual query layers, distributed MVCC with hybrid logical clocks, automatic tablet splitting and rebalancing, xCluster multi-region replication, and production considerations for hotspots, connection pooling, and schema design.