The Service Decomposition Playbook: Extracting Services Without Breaking Production

You have already decided to extract a service. Maybe you followed the signals: independent deployment pressure, divergent scaling profiles, a team boundary that keeps causing merge conflicts. The decision is made. Now comes the part nobody warns you about: actually doing it without taking down production.

Most “microservices migration” content stops at the architecture diagram. Draw some boxes, add some arrows, wave hands about eventual consistency. That is the easy part. The hard part is the six weeks of dual-writes, the data reconciliation scripts you run at 2 AM, and the moment you realize your “clean boundary” shares 14 database tables with three other modules.

This is the execution playbook. No architecture philosophy. Just the steps, the traps, and the code.

Step 1: Map Dependencies Before You Touch Anything

Before writing a single line of migration code, build a dependency map of the module you want to extract. Not the one in your head. The real one.

Start with database tables. Which tables does this module own? Which does it read from that belong to other modules? Which tables do other modules read from that belong to this one?

// Script to scan your codebase for cross-module table access
// Run this against your query layer to find hidden coupling

interface TableDependency {
  table: string;
  module: string;
  accessType: "read" | "write";
  queryLocations: string[];
}

function scanForTableAccess(
  sourceDir: string,
  targetTables: string[]
): TableDependency[] {
  // Parse all query files, ORM models, and raw SQL
  // Flag any access to targetTables from outside the owning module
  const dependencies: TableDependency[] = [];

  for (const file of walkTypeScriptFiles(sourceDir)) {
    const content = readFileSync(file, "utf-8");
    for (const table of targetTables) {
      const patterns = [
        new RegExp(`from\\s+['"]?${table}['"]?`, "gi"),
        new RegExp(`join\\s+['"]?${table}['"]?`, "gi"),
        new RegExp(`\\.${table}\\.`, "gi"), // ORM access
        new RegExp(`into\\s+['"]?${table}['"]?`, "gi"),
      ];

      for (const pattern of patterns) {
        if (pattern.test(content)) {
          const module = extractModuleName(file);
          dependencies.push({
            table,
            module,
            accessType: content.match(/insert|update|delete/i)
              ? "write"
              : "read",
            queryLocations: [file],
          });
        }
      }
    }
  }

  return dependencies;
}

The output will surprise you. Tables you thought were private to the billing module turn out to be joined by the reporting module, the admin dashboard, and a cron job someone wrote two years ago. Each of these is a coupling point you must resolve before extraction.

Classify every dependency into one of three categories:

1. Owns and should keep: Tables that move with the service.
2. Reads from others: Replace with API calls or events after extraction.
3. Others read from this: You need to expose this data through the new service’s API.

Do not skip this step. Every surprise dependency you miss becomes a production incident during cutover.

Step 2: Design the API Contract First

Write the API contract before you write the service. Not after. Not “iteratively.” Before.

This contract is the thing that every consumer will depend on. Getting it wrong means coordinated changes across multiple services, which is the exact problem you were trying to escape.

// contracts/billing-service.ts
// This file is the source of truth. Service and consumers both import from here.

export interface CreateInvoiceRequest {
  accountId: string;
  lineItems: InvoiceLineItem[];
  currency: string;
  dueDate: string; // ISO 8601
  idempotencyKey: string;
}

export interface InvoiceLineItem {
  description: string;
  quantity: number;
  unitPriceCents: number;
  taxRate: number;
}

export interface CreateInvoiceResponse {
  invoiceId: string;
  status: "draft" | "pending" | "sent";
  totalCents: number;
  createdAt: string;
}

export interface GetInvoiceResponse {
  invoiceId: string;
  accountId: string;
  status: "draft" | "pending" | "sent" | "paid" | "void";
  lineItems: InvoiceLineItem[];
  totalCents: number;
  currency: string;
  dueDate: string;
  createdAt: string;
  paidAt: string | null;
}

// Error contract: consumers need to handle these explicitly
export type BillingServiceError =
  | { code: "ACCOUNT_NOT_FOUND"; accountId: string }
  | { code: "DUPLICATE_IDEMPOTENCY_KEY"; existingInvoiceId: string }
  | { code: "INVALID_LINE_ITEMS"; details: string[] };

Three rules for the contract:

Include an idempotency key on every write operation. Network calls fail. Retries happen. Without idempotency, you will create duplicate invoices and your finance team will find you.

Version from day one. Not because you will need it immediately, but because adding versioning later requires coordinating every consumer at once. A simple path prefix (/v1/invoices) costs nothing upfront and saves weeks later.

Define error types explicitly. “500 Internal Server Error” is not a contract. Your consumers need to know whether to retry, show a user-facing message, or page someone.

Step 3: Build the Strangler Fig, Not a Big Bang

The strangler fig pattern is not optional for production systems. You cannot rewrite and swap in one deployment. You need a transition period where both the old and new code paths exist, and you can route traffic between them.

Here is the implementation pattern:

// middleware/billing-router.ts
// Routes billing requests to either the monolith or the new service

interface RoutingConfig {
  serviceUrl: string;
  enabled: boolean;
  rolloutPercentage: number; // 0-100
  allowlist: string[]; // account IDs for canary testing
}

async function routeBillingRequest(
  req: Request,
  config: RoutingConfig
): Promise<Response> {
  if (!config.enabled) {
    return handleLocally(req);
  }

  const accountId = extractAccountId(req);

  // Phase 1: Canary with specific accounts
  if (config.allowlist.includes(accountId)) {
    return forwardToService(req, config.serviceUrl);
  }

  // Phase 2: Percentage-based rollout
  if (config.rolloutPercentage > 0) {
    const hash = hashAccountId(accountId);
    const bucket = hash % 100;
    if (bucket < config.rolloutPercentage) {
      return forwardToService(req, config.serviceUrl);
    }
  }

  // Default: handle in monolith
  return handleLocally(req);
}

async function forwardToService(
  req: Request,
  serviceUrl: string
): Promise<Response> {
  const startTime = Date.now();

  try {
    const response = await fetch(`${serviceUrl}${req.url}`, {
      method: req.method,
      headers: req.headers,
      body: req.body,
    });

    recordMetric("billing_service_request", {
      status: response.status,
      duration: Date.now() - startTime,
      target: "new_service",
    });

    return response;
  } catch (error) {
    recordMetric("billing_service_error", {
      error: error instanceof Error ? error.message : "unknown",
      target: "new_service",
    });

    // Fallback to monolith on failure
    return handleLocally(req);
  }
}

The rollout sequence matters:

1. Deploy the new service with zero traffic. Verify it starts, connects to its database, and passes health checks.
2. Route 1-2 internal test accounts. Verify correct behavior with real data.
3. Route 5% of traffic. Watch error rates, latency p99, and business metrics for 48 hours.
4. Increase to 25%, then 50%, then 100%. Each step gets at least 24 hours of observation.
5. Only after 100% has been stable for a week do you remove the old code path.

Step 5 is where teams get sloppy. The old code stays “just in case” for months, accumulating drift. Set a calendar reminder. Remove it.

Step 4: Solve the Data Problem

Data migration is where service extractions actually fail. The code routing is straightforward. Splitting a shared database without losing data, creating inconsistencies, or causing downtime is not.

The Dual-Write Trap

The naive approach: write to both the old and new databases during migration. This is almost always wrong.

// DO NOT DO THIS without understanding the failure modes

async function createInvoice(data: CreateInvoiceRequest): Promise<Invoice> {
  // Write to old database
  const oldResult = await oldDb.invoices.insert(data);

  // Write to new database
  const newResult = await newDb.invoices.insert(data);

  // What happens when one succeeds and the other fails?
  // What happens when they succeed but return different IDs?
  // What happens when a third write to old DB happens between these two?

  return oldResult;
}

Dual writes without a coordination mechanism guarantee data divergence. Instead, use one of these patterns:

Pattern A: Change Data Capture (CDC)

The old database remains the source of truth during migration. A CDC pipeline (Debezium, or a custom WAL reader) streams changes to the new database in near-real-time.

// CDC consumer that replicates billing data to the new service's database

interface CDCEvent {
  table: string;
  operation: "INSERT" | "UPDATE" | "DELETE";
  before: Record<string, unknown> | null;
  after: Record<string, unknown> | null;
  timestamp: string;
  transactionId: string;
}

async function handleCDCEvent(event: CDCEvent): Promise<void> {
  if (event.table !== "invoices") return;

  switch (event.operation) {
    case "INSERT":
      await newDb.invoices.insert(
        transformToNewSchema(event.after!)
      );
      break;
    case "UPDATE":
      await newDb.invoices.update(
        { oldId: event.after!.id },
        transformToNewSchema(event.after!)
      );
      break;
    case "DELETE":
      await newDb.invoices.softDelete({ oldId: event.before!.id });
      break;
  }

  // Track replication lag
  const lag = Date.now() - new Date(event.timestamp).getTime();
  recordMetric("cdc_replication_lag_ms", lag);

  if (lag > 5000) {
    emitAlert("CDC replication lag exceeds 5 seconds");
  }
}

Pattern B: Synchronous Replication via the Application

If CDC infrastructure is too heavy for your scale, replicate at the application layer using a transactional outbox.

// Outbox pattern: write to old DB + outbox in one transaction
// A separate worker processes the outbox and writes to the new DB

async function createInvoiceWithOutbox(
  data: CreateInvoiceRequest
): Promise<Invoice> {
  return await oldDb.transaction(async (tx) => {
    // Write the actual record
    const invoice = await tx.invoices.insert(data);

    // Write to outbox in the same transaction
    await tx.outbox.insert({
      aggregateType: "invoice",
      aggregateId: invoice.id,
      eventType: "CREATED",
      payload: JSON.stringify(invoice),
      processedAt: null,
    });

    return invoice;
  });
}

// Outbox processor runs on a schedule
async function processOutbox(): Promise<void> {
  const pending = await oldDb.outbox.findMany({
    where: { processedAt: null },
    orderBy: { createdAt: "asc" },
    limit: 100,
  });

  for (const entry of pending) {
    try {
      await newDb.invoices.upsert(
        JSON.parse(entry.payload)
      );
      await oldDb.outbox.update(
        { id: entry.id },
        { processedAt: new Date().toISOString() }
      );
    } catch (error) {
      await oldDb.outbox.update(
        { id: entry.id },
        { lastError: String(error), retryCount: entry.retryCount + 1 }
      );
    }
  }
}

Data Reconciliation

Whichever replication approach you use, you need a reconciliation job. Trust nothing. Verify everything.

// Run this daily during migration. It will find problems you did not expect.

async function reconcileInvoices(): Promise<ReconciliationReport> {
  const oldInvoices = await oldDb.invoices.findMany({
    where: { updatedAt: { gte: oneDayAgo() } },
  });

  const discrepancies: Discrepancy[] = [];

  for (const oldInvoice of oldInvoices) {
    const newInvoice = await newDb.invoices.findByOldId(oldInvoice.id);

    if (!newInvoice) {
      discrepancies.push({
        type: "MISSING_IN_NEW",
        oldId: oldInvoice.id,
        severity: "critical",
      });
      continue;
    }

    // Compare every field that matters
    const fieldChecks = [
      ["totalCents", oldInvoice.totalCents, newInvoice.totalCents],
      ["status", oldInvoice.status, newInvoice.status],
      ["accountId", oldInvoice.accountId, newInvoice.accountId],
    ] as const;

    for (const [field, oldVal, newVal] of fieldChecks) {
      if (oldVal !== newVal) {
        discrepancies.push({
          type: "FIELD_MISMATCH",
          oldId: oldInvoice.id,
          field,
          oldValue: String(oldVal),
          newValue: String(newVal),
          severity: field === "totalCents" ? "critical" : "warning",
        });
      }
    }
  }

  return {
    checked: oldInvoices.length,
    discrepancies,
    criticalCount: discrepancies.filter((d) => d.severity === "critical").length,
  };
}

Run reconciliation from day one of the migration, not the day before cutover. Problems compound. Finding them early is the difference between a fix and an incident.

Step 5: Plan the Cutover

The cutover is a single, planned event. Treat it like a deployment to production, because it is one.

Write a runbook. Not a wiki page with general guidance. A step-by-step script with commands, expected outputs, and decision points.

## Billing Service Cutover Runbook

### Pre-cutover (T-24h)
- [ ] Reconciliation report shows 0 critical discrepancies for 72h
- [ ] New service handles 100% of read traffic for 7 days
- [ ] p99 latency on new service < 200ms (current: ___ ms)
- [ ] Error rate on new service < 0.1% (current: ___%)
- [ ] On-call team briefed, runbook reviewed
- [ ] Rollback procedure tested in staging

### Cutover (T-0)
- [ ] Announce maintenance window in #engineering
- [ ] Set routing config: writes to new service (100%)
- [ ] Monitor dashboard for 15 minutes
- [ ] Verify: new invoices appear in new DB only
- [ ] Verify: old consumers reading via API (not direct DB)

### Post-cutover (T+1h)
- [ ] Run reconciliation: confirm no new writes to old DB
- [ ] Check downstream systems: reporting, analytics, admin
- [ ] Confirm: no direct queries hitting old billing tables

### Rollback trigger
Any of these means immediate rollback:
- Error rate > 1% for 5 minutes
- p99 latency > 500ms for 5 minutes
- Data reconciliation shows new discrepancies
- Any 5xx from billing API affecting customer-facing flows

The Rollback Plan

You need a rollback plan that works in under five minutes. If your rollback requires a database migration, a code deployment, or “coordinating with the team,” it is not a rollback plan.

// Rollback is a config change, not a deployment

interface CutoverConfig {
  writeTarget: "monolith" | "new_service";
  readTarget: "monolith" | "new_service" | "both";
  rollbackEnabled: boolean;
}

async function rollback(config: CutoverConfig): Promise<void> {
  // Step 1: Route all writes back to monolith
  await updateConfig({
    ...config,
    writeTarget: "monolith",
    readTarget: "monolith",
  });

  // Step 2: Verify monolith is handling requests
  const healthCheck = await fetch(`${monolithUrl}/health/billing`);
  if (!healthCheck.ok) {
    emitAlert("CRITICAL: Rollback health check failed");
    return;
  }

  // Step 3: Replay any writes that went to new service during cutover
  // These are captured in the new service's outbox
  const missedWrites = await newDb.outbox.findMany({
    where: {
      createdAt: { gte: cutoverStartTime },
      replayedAt: null,
    },
  });

  for (const write of missedWrites) {
    await oldDb.invoices.upsert(JSON.parse(write.payload));
    await newDb.outbox.update(
      { id: write.id },
      { replayedAt: new Date().toISOString() }
    );
  }

  emitAlert(`Rollback complete. ${missedWrites.length} writes replayed.`);
}

The key insight: your new service must also maintain an outbox during the cutover period. If you need to roll back, those writes need to flow back to the monolith. Without this, you lose data.

Step 6: Operational Readiness

A service that works is not a service that is ready for production. Before you declare the extraction complete, verify operational readiness.

// Operational readiness checklist as code
// Run this before declaring the service "done"

interface ReadinessCheck {
  name: string;
  check: () => Promise<boolean>;
  severity: "blocker" | "warning";
}

const readinessChecks: ReadinessCheck[] = [
  {
    name: "Health endpoint responds",
    check: async () => {
      const res = await fetch(`${serviceUrl}/health`);
      return res.ok;
    },
    severity: "blocker",
  },
  {
    name: "Metrics are being scraped",
    check: async () => {
      const metrics = await fetch(`${serviceUrl}/metrics`);
      const body = await metrics.text();
      return body.includes("http_request_duration_seconds");
    },
    severity: "blocker",
  },
  {
    name: "Structured logging configured",
    check: async () => {
      const logs = await fetchRecentLogs(serviceUrl);
      return logs.every((log) => isValidJSON(log));
    },
    severity: "blocker",
  },
  {
    name: "Alerts configured for error rate",
    check: async () => {
      const alerts = await getConfiguredAlerts(serviceUrl);
      return alerts.some((a) => a.metric === "error_rate");
    },
    severity: "blocker",
  },
  {
    name: "Runbook linked in alert definitions",
    check: async () => {
      const alerts = await getConfiguredAlerts(serviceUrl);
      return alerts.every((a) => a.runbookUrl !== undefined);
    },
    severity: "warning",
  },
  {
    name: "Circuit breaker configured for downstream calls",
    check: async () => {
      const config = await getServiceConfig(serviceUrl);
      return config.circuitBreaker.enabled;
    },
    severity: "blocker",
  },
  {
    name: "Graceful shutdown handles in-flight requests",
    check: async () => {
      // Send a slow request, then SIGTERM the service
      // Verify the request completes before shutdown
      return await testGracefulShutdown(serviceUrl);
    },
    severity: "blocker",
  },
  {
    name: "Database connection pool sized correctly",
    check: async () => {
      const poolConfig = await getServiceConfig(serviceUrl);
      // Pool should be sized for expected concurrency + headroom
      return poolConfig.db.poolSize >= 10 && poolConfig.db.poolSize <= 50;
    },
    severity: "warning",
  },
];

async function runReadinessChecks(): Promise<void> {
  const results: Array<{ name: string; passed: boolean; severity: string }> = [];

  for (const check of readinessChecks) {
    try {
      const passed = await check.check();
      results.push({ name: check.name, passed, severity: check.severity });
    } catch {
      results.push({ name: check.name, passed: false, severity: check.severity });
    }
  }

  const blockers = results.filter((r) => !r.passed && r.severity === "blocker");
  const warnings = results.filter((r) => !r.passed && r.severity === "warning");

  console.log(`Passed: ${results.filter((r) => r.passed).length}/${results.length}`);

  if (blockers.length > 0) {
    console.error("BLOCKERS (must fix before cutover):");
    blockers.forEach((b) => console.error(`  - ${b.name}`));
  }

  if (warnings.length > 0) {
    console.warn("WARNINGS (should fix, not blocking):");
    warnings.forEach((w) => console.warn(`  - ${w.name}`));
  }
}

The Timeline Nobody Talks About

Teams consistently underestimate how long a service extraction takes. Here is a realistic breakdown for a medium-complexity module (owns 5-10 tables, 3-4 consumer modules):

• Dependency mapping and contract design: 1-2 weeks
• Building the new service with tests: 2-3 weeks
• Data replication pipeline and reconciliation: 2-3 weeks
• Strangler fig routing and canary testing: 1-2 weeks
• Progressive rollout (5% to 100%): 2-4 weeks
• Old code path removal and cleanup: 1 week

Total: 9-15 weeks for a single service extraction. This is normal. If someone estimates two weeks, they are either extracting a trivial module or they are going to have a bad time.

The reason it takes this long is not the code. It is the confidence-building. Each phase exists to prove that the next phase is safe. Skip a phase and you are gambling with production data.

Common Failure Modes

After watching (and participating in) several service extractions, these patterns keep showing up:

Shared database lingers. The service is “extracted” but still reads from the monolith’s database directly. This is not a service. This is a monolith with extra network hops. If the service cannot function when the monolith database is unreachable, you have not extracted anything.

Contract changes require synchronized deploys. If changing the billing service API requires deploying the billing service and two consumers simultaneously, your contract design failed. Use additive changes, deprecation periods, and versioning.

No ownership transfer. The service exists but the same team still maintains both the monolith and the service. Service extraction without team extraction just doubles the maintenance surface. Assign clear ownership before you start.

Testing the happy path only. Your integration tests cover the case where the service responds correctly. Do they cover the case where the service is down? Where it responds in 30 seconds? Where it returns an unexpected error code? Failure modes in distributed systems are not edge cases. They are Tuesday.

When You Are Done

You are done when:

1. The new service handles 100% of traffic for its domain.
2. No direct database access from outside the service boundary.
3. Reconciliation shows zero discrepancies for two weeks.
4. The old code path is deleted (not commented out, not behind a flag, deleted).
5. On-call rotation includes the new service with tested runbooks.
6. The team that owns the service can deploy it independently.

If any of these are not true, the extraction is not complete. It is in progress. Call it what it is, because “in progress” extractions that get declared “done” become the next generation of tech debt.

Service extraction is not glamorous work. There is no conference talk in “I spent three months writing reconciliation scripts.” But it is the work that makes the architecture actually function, instead of just looking good on a whiteboard.