Idempotency Keys for Reliable Backend APIs: Stop Duplicate Charges and Double Writes

Most backend outages are noisy. Duplicate write bugs are quiet.

A user taps “Pay” twice because their mobile network stutters. Your client retries after a timeout. Your gateway retries after a 502. Your queue worker restarts and replays. Suddenly one intent became two charges, three orders, or ten webhook deliveries.

This is exactly the class of failures idempotency keys are meant to prevent.

If your API performs side effects (payments, order creation, provisioning, email sends), idempotency is not a nice-to-have. It is a hard reliability requirement.

What Idempotency Actually Means (and What It Doesn’t)

Idempotency means: the same operation request can be executed multiple times with the same effect as one execution.

In API practice, that means:

• Client sends a unique key with the request (for example Idempotency-Key).
• Server stores the first result for that key.
• Any retry with the same key returns the same response instead of re-running side effects.

Two clarifications that matter:

1. Idempotency is not exactly-once processing. Distributed systems rarely guarantee exactly once end to end. Idempotency gives you effectively-once outcomes for a defined scope.

2. Idempotency is scoped. A key is usually unique per endpoint + actor (tenant/user/account), not globally across your entire platform.

When You Must Use Idempotency Keys

Use idempotency for any endpoint where duplicate effects are unacceptable:

• Payment charge creation
• Subscription creation / plan changes
• Order placement
• Invoice generation
• Credit/debit balance mutation
• Provisioning cloud resources
• Sending non-repeatable outbound actions (SMS with one-time code, billing emails)

If the action is read-only (GET) or naturally safe (PUT replacing the same resource by id), key-based idempotency may be unnecessary.

The Baseline Protocol

At minimum, implement this flow:

1. Client generates UUID key per user intent.
2. Server checks key store.
3. If key is new: lock/claim it, run business logic once, persist response.
4. If key exists and completed: return stored response.
5. If key exists and in-progress: return 409 or 202 with retry guidance.

Example request:

POST /api/v1/payments
Idempotency-Key: 2d8bfc77-a6a0-4e34-bf75-cc4a4f4c91f9
Content-Type: application/json

{
  "customerId": "cus_123",
  "amount": 4900,
  "currency": "USD",
  "paymentMethodId": "pm_abc"
}

Critical point: the same key must represent the same request payload. If the payload changes, reject.

Request Fingerprinting: Prevent Key Reuse Abuse

Without fingerprinting, a buggy or malicious client can reuse a key for a different payload and cause undefined behavior.

Store a normalized request hash with the key:

import crypto from "node:crypto";

type PaymentInput = {
  customerId: string;
  amount: number;
  currency: string;
  paymentMethodId: string;
};

function normalizePaymentInput(input: PaymentInput): string {
  // Canonicalize shape + field order to avoid hash drift
  return JSON.stringify({
    customerId: input.customerId,
    amount: input.amount,
    currency: input.currency,
    paymentMethodId: input.paymentMethodId,
  });
}

function fingerprint(input: PaymentInput): string {
  return crypto
    .createHash("sha256")
    .update(normalizePaymentInput(input))
    .digest("hex");
}

On retry:

• same key + same fingerprint → replay stored response
• same key + different fingerprint → 409 Conflict (or 422) and never execute

This single rule eliminates a lot of edge-case corruption.

Storage Design: Redis vs Postgres

Both work. Pick based on your durability and latency requirements.

Redis Pattern (fast path)

Use when you need low latency and can tolerate TTL-based cache semantics.

Suggested value shape:

{
  "status": "completed",
  "requestHash": "...",
  "statusCode": 201,
  "responseBody": { "paymentId": "pay_123", "status": "succeeded" },
  "createdAt": "2026-03-08T10:00:00.000Z",
  "expiresAt": "2026-03-09T10:00:00.000Z"
}

Use SET key value NX EX <ttl> to claim first writer atomically.

Postgres Pattern (durable path)

Use when correctness/auditability matters (payments, financial writes).

CREATE TABLE api_idempotency (
  scope TEXT NOT NULL,
  idempotency_key TEXT NOT NULL,
  request_hash TEXT NOT NULL,
  status TEXT NOT NULL CHECK (status IN ('in_progress', 'completed', 'failed')),
  status_code INT,
  response_body JSONB,
  error_body JSONB,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  expires_at TIMESTAMPTZ NOT NULL,
  PRIMARY KEY (scope, idempotency_key)
);

CREATE INDEX idx_api_idempotency_expires_at ON api_idempotency (expires_at);

Unique PK gives you natural race protection across app instances.

Race Conditions: The Failure Most Teams Miss

Naive flow:

1. SELECT key not found
2. run side effect
3. INSERT key

Under concurrency, two instances both pass step 1 and both execute step 2.

Correct approach: claim before side effects.

Postgres example with transaction:

import { Pool } from "pg";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });

async function claimIdempotency(
  scope: string,
  key: string,
  requestHash: string,
  ttlHours = 24,
) {
  const client = await pool.connect();
  try {
    await client.query("BEGIN");

    const insert = await client.query(
      `INSERT INTO api_idempotency (scope, idempotency_key, request_hash, status, expires_at)
       VALUES ($1, $2, $3, 'in_progress', now() + ($4 || ' hours')::interval)
       ON CONFLICT (scope, idempotency_key) DO NOTHING
       RETURNING scope, idempotency_key`,
      [scope, key, requestHash, String(ttlHours)],
    );

    if (insert.rowCount === 1) {
      await client.query("COMMIT");
      return { state: "claimed" as const };
    }

    const existing = await client.query(
      `SELECT request_hash, status, status_code, response_body, error_body
       FROM api_idempotency
       WHERE scope = $1 AND idempotency_key = $2`,
      [scope, key],
    );

    await client.query("COMMIT");
    return { state: "existing" as const, row: existing.rows[0] };
  } catch (err) {
    await client.query("ROLLBACK");
    throw err;
  } finally {
    client.release();
  }
}

The first requester claims the key. Everyone else becomes a replay path.

End-to-End Handler Pattern (TypeScript)

This is the production skeleton to copy:

import type { Request, Response } from "express";

export async function createPaymentHandler(req: Request, res: Response) {
  const idempotencyKey = req.header("Idempotency-Key");
  if (!idempotencyKey) {
    return res.status(400).json({ error: "Missing Idempotency-Key header" });
  }

  const scope = `tenant:${req.auth.tenantId}:endpoint:create_payment`;
  const payload = req.body as {
    customerId: string;
    amount: number;
    currency: string;
    paymentMethodId: string;
  };

  const requestHash = fingerprint(payload);
  const claim = await claimIdempotency(scope, idempotencyKey, requestHash);

  // Existing key path
  if (claim.state === "existing") {
    const row = claim.row;

    if (row.request_hash !== requestHash) {
      return res.status(409).json({ error: "Idempotency key reused with different payload" });
    }

    if (row.status === "completed") {
      return res.status(row.status_code ?? 200).json(row.response_body);
    }

    if (row.status === "failed") {
      return res.status(row.status_code ?? 400).json(row.error_body ?? { error: "Request failed" });
    }

    // still running on another worker
    return res.status(409).json({
      error: "Request already in progress",
      retryAfterSeconds: 2,
    });
  }

  // Claimed path: execute side effects once
  try {
    const payment = await paymentsService.createCharge({
      tenantId: req.auth.tenantId,
      customerId: payload.customerId,
      amount: payload.amount,
      currency: payload.currency,
      paymentMethodId: payload.paymentMethodId,
    });

    await markIdempotencyCompleted(scope, idempotencyKey, 201, {
      paymentId: payment.id,
      status: payment.status,
      amount: payment.amount,
      currency: payment.currency,
    });

    return res.status(201).json({
      paymentId: payment.id,
      status: payment.status,
      amount: payment.amount,
      currency: payment.currency,
    });
  } catch (err) {
    const errorBody = { error: "Payment failed" };
    await markIdempotencyFailed(scope, idempotencyKey, 502, errorBody);
    return res.status(502).json(errorBody);
  }
}

The important operational behavior: failed attempts are stored too. Otherwise retries can hammer a downstream that is already failing and produce inconsistent user-visible outcomes.

TTL Strategy: How Long to Keep Keys

There is no universal number. Use domain-based windows:

• payments/orders: 24–72 hours
• internal provisioning: 6–24 hours
• webhook consumer dedupe: 24 hours minimum

Too short: retries after a network partition can duplicate side effects. Too long: unbounded growth and storage cost.

Practical policy:

• keep active keys in primary store
• run periodic cleanup on expires_at
• archive high-risk operations (payments) to long-term audit table/log

Status Codes and Client Contract

Be explicit so client teams can implement correct retry logic:

• 201/200: first success or replay success
• 409: key conflict (different payload) or in-progress request
• 400: missing/invalid key
• 429: optional, if you rate-limit key creation abuse

Return same body and status for replayed success whenever possible. Predictability reduces client-side branching bugs.

Idempotency Across Async Boundaries (Queues + Webhooks)

HTTP idempotency is only one layer. You also need dedupe in asynchronous flows.

Queue Workers

Use operation id as dedupe key in worker storage before side effect execution.

Webhook Consumers

Most providers include event IDs. Store processed event IDs with TTL and reject duplicates.

async function handleStripeEvent(event: { id: string; type: string; data: any }) {
  const seen = await redis.set(`stripe:event:${event.id}`, "1", "NX", "EX", 86400);
  if (seen === null) {
    // duplicate delivery, already handled
    return;
  }

  await processEvent(event);
}

Without this, your API can be idempotent while your downstream effects still duplicate.

Observability: What to Track in Production

Track these metrics from day one:

• idempotency.claimed (new keys)
• idempotency.replayed (duplicate retries served from store)
• idempotency.conflict (same key, different payload)
• idempotency.in_progress (concurrency pressure)
• idempotency.storage_error
• p95 latency for first-run vs replay path

And log with structured fields:

• idempotency_key
• scope
• request_hash
• decision (claimed|replayed|conflict|in_progress)

This lets you detect buggy SDKs early (you’ll see replay/conflict spikes immediately).

Common Implementation Mistakes

1. Using one global key namespace. Causes accidental collisions across endpoints/tenants.

2. No request hash check. Lets changed payloads reuse old keys.

3. Claiming key after side effect. Guarantees race duplicates under load.

4. Not persisting failed outcomes. Makes retries nondeterministic during partial outages.

5. No async dedupe. HTTP layer looks correct while workers/webhooks duplicate effects.

Closing

Idempotency keys are one of the highest-leverage backend reliability patterns because they convert retry chaos into deterministic behavior.

The production version is simple in principle:

• scope key correctly,
• claim before side effects,
• verify payload fingerprint,
• persist outcome,
• replay deterministically.

Get this right and you eliminate an entire class of expensive incidents: duplicate charges, double orders, repeated provisioning, and replay storms after transient failures.

For teams building payment or transactional APIs at Let’s Build Solutions, this is baseline engineering hygiene, not an advanced feature.