Designing an AI Agent Cost Governance System: Token Budgets, Spend Caps, and Automated Circuit Breakers for Production LLM Deployments

Uber’s engineering org reportedly burned through its entire 2026 AI tooling budget by April. Multiple teams had independently enabled AI coding assistants at $500 to $2,000 per engineer per month. Nobody owned a consolidated view of spend. Nobody had set a hard limit. By the time finance flagged it, the damage was done.

That story is repeating at smaller scale across dozens of startups and growth-stage companies right now. The pattern is consistent: AI features get added to production, agents run on autopilot, and nobody notices the cost trajectory until an invoice lands.

The engineering problem behind all of it is the same: most teams treat AI costs as an observability problem when they are actually an enforcement problem. Dashboards that show you where the money went after it is gone are useful for postmortems. They do not stop the next incident.

This article covers how to build a cost governance system that enforces budgets in real time, not just reports on them after the fact.

The $47K Agent Loop

Before the architecture, a concrete failure mode worth understanding.

A team had four LangChain agents working in sequence: a research agent, a summarization agent, a critic agent, and a revision agent. Each agent passed its full output to the next, and the critic agent could route back to the research agent for additional context.

The loop ran for 264 hours before someone checked the billing dashboard. The root cause was O(n²) context accumulation: each pass through the loop appended the previous agent’s output to the next agent’s context window. By hour 48, each individual completion was consuming 80,000+ tokens. The critic agent kept routing back to research because it was evaluating summaries that were themselves getting incoherent from context overflow.

Total spend: $47,000. The feature was a background enrichment job that nobody was actively monitoring.

Three things would have stopped this:

1. A per-agent token budget that halted execution when a single agent call exceeded a threshold
2. A loop iteration cap that stopped the orchestration after N cycles
3. A total spend cap for the task that triggered a circuit breaker and paged the on-call engineer

None of those existed. The team had logging. They did not have enforcement.

Monitoring Is Not Governance

The monitoring vs. enforcement distinction is the most important concept in this entire topic, so it is worth making explicit before touching code.

Monitoring means you record what happened and alert when thresholds are crossed. You can build excellent dashboards that show cost per feature, cost per team, cost trend over time, and P90/P99 of token usage per agent. All of that is valuable. None of it prevents a runaway loop from running for 264 hours.

Enforcement means the system cannot exceed the budget. It is a gate, not a gauge. When the budget is exhausted, the operation stops. The agent does not continue calling the API. The circuit opens.

Most LLM observability tooling sold to engineering teams is monitoring. It captures traces, aggregates metrics, and emails you reports. The enforcement layer you have to build yourself, because it lives in your call path, not in a third-party dashboard.

The architecture has two planes: a data plane (the enforcement middleware that runs on every LLM call) and a control plane (the budget store, allocation logic, and circuit breaker state machine).

The Budget Store

Start with how budgets are represented. You need a structure that supports three levels of granularity: per-agent (the tightest constraint), per-feature (a collection of agents working together), and per-team (the organizational unit that owns the spend).

interface BudgetPolicy {
  id: string;
  scope: "agent" | "feature" | "team";
  scopeId: string;
  periodStart: Date;
  periodEnd: Date;
  limitCents: number;
  spentCents: number;
  warningThresholdPct: number; // alert at this percent, default 80
  hardCapEnabled: boolean;     // if false, allow overage but alert
  circuitBreakerEnabled: boolean;
}

interface BudgetAllocation {
  teamId: string;
  totalLimitCents: number;
  features: {
    featureId: string;
    limitCents: number;
    agents: {
      agentId: string;
      perCallLimitCents: number;    // max spend per single call
      periodLimitCents: number;     // max spend over the period
      maxIterations?: number;       // for loop-based agents
      maxContextTokens?: number;    // per-call context cap
    }[];
  }[];
}

The perCallLimitCents field is doing important work here. A period budget tells you when to stop a feature over time. A per-call limit stops a single runaway completion: if one agent call is about to consume 200,000 tokens, reject it before the API call goes out.

Store this in a database with a fast read path. You need to check budgets on every LLM call, which means the budget lookup needs to be sub-5ms for a 200ms inference request to remain under its SLA. Redis with Lua scripts for atomic increment-and-check works well. Postgres with optimistic locking works at lower call rates.

The Enforcement Middleware

This is the gate in your call path. Every LLM call routes through it before hitting the provider.

interface LLMCallContext {
  agentId: string;
  featureId: string;
  teamId: string;
  requestId: string;
  estimatedInputTokens: number;
  estimatedOutputTokens: number;
}

interface BudgetCheckResult {
  allowed: boolean;
  reason?: "per_call_limit" | "period_limit" | "circuit_open" | "team_exhausted";
  remainingCents: number;
  spentCents: number;
}

async function checkBudget(
  ctx: LLMCallContext,
  model: string
): Promise<BudgetCheckResult> {
  const estimatedCostCents = estimateCost(
    ctx.estimatedInputTokens,
    ctx.estimatedOutputTokens,
    model
  );

  const agentPolicy = await budgetStore.getPolicy("agent", ctx.agentId);

  // Per-call hard cap: reject before the API call goes out
  if (agentPolicy && estimatedCostCents > agentPolicy.perCallLimitCents) {
    return {
      allowed: false,
      reason: "per_call_limit",
      remainingCents: agentPolicy.limitCents - agentPolicy.spentCents,
      spentCents: agentPolicy.spentCents,
    };
  }

  // Check circuit breaker state
  const circuitState = await circuitBreakerStore.getState(ctx.agentId);
  if (circuitState === "open") {
    return {
      allowed: false,
      reason: "circuit_open",
      remainingCents: 0,
      spentCents: agentPolicy?.spentCents ?? 0,
    };
  }

  // Period budget check (atomic read-modify-write)
  const periodCheck = await budgetStore.reserveAndCheck(
    ctx.agentId,
    ctx.featureId,
    ctx.teamId,
    estimatedCostCents
  );

  return periodCheck;
}

async function recordActualSpend(
  ctx: LLMCallContext,
  actualCostCents: number
): Promise<void> {
  // Correct for estimation error after the call completes
  await budgetStore.adjustReservation(
    ctx.agentId,
    ctx.featureId,
    ctx.teamId,
    actualCostCents
  );
}

The two-phase approach (reserve an estimate, then adjust to actual) prevents you from blocking calls based on overestimates while still protecting against going over budget. The reservation step uses an atomic increment in Redis. The adjustment step corrects for the difference between estimated and actual token counts.

The middleware wraps your LLM client:

async function callLLMWithBudgetEnforcement(
  ctx: LLMCallContext,
  messages: LLMMessage[],
  model: string,
  options: InferenceOptions
): Promise<InferenceResponse> {
  const budgetCheck = await checkBudget(ctx, model);

  if (!budgetCheck.allowed) {
    await emitBudgetEvent({
      type: "BUDGET_BLOCKED",
      ctx,
      reason: budgetCheck.reason,
      remainingCents: budgetCheck.remainingCents,
    });

    throw new BudgetExhaustedError(
      `LLM call blocked: ${budgetCheck.reason}. ` +
      `Agent ${ctx.agentId} remaining budget: ${budgetCheck.remainingCents}¢`
    );
  }

  const start = Date.now();
  let response: InferenceResponse;

  try {
    response = await llmClient.complete(messages, model, options);
  } catch (err) {
    // Release the reservation if the call failed
    await budgetStore.releaseReservation(ctx.agentId, ctx.featureId, ctx.teamId);
    throw err;
  }

  await recordActualSpend(ctx, response.usage.estimatedCostCents);
  await maybeUpdateCircuitBreaker(ctx, response);

  return response;
}

The Circuit Breaker

A circuit breaker for cost is different from a circuit breaker for availability. You are not tracking error rates: you are tracking spend velocity relative to expected patterns.

Three conditions warrant opening the circuit:

1. Spend rate anomaly: The agent is spending 5x its historical average per minute. This catches loops that are accelerating.
2. Period budget at threshold: The agent has consumed 95% of its period budget and is still running.
3. Per-call cost escalation: Individual calls are growing in cost over successive iterations, which is the signature of O(n²) context accumulation.

interface CircuitBreakerState {
  agentId: string;
  state: "closed" | "open" | "half_open";
  openedAt?: Date;
  openReason?: string;
  callsInWindow: number;
  spendInWindowCents: number;
  lastCallCostCents: number;
  previousCallCostCents: number;
  windowStartedAt: Date;
}

const WINDOW_SECONDS = 60;
const SPEND_VELOCITY_MULTIPLIER = 5;
const CONTEXT_ESCALATION_RATIO = 2.0; // current call cost / previous call cost

async function maybeUpdateCircuitBreaker(
  ctx: LLMCallContext,
  response: InferenceResponse
): Promise<void> {
  const state = await circuitBreakerStore.update(ctx.agentId, {
    lastCallCostCents: response.usage.estimatedCostCents,
    callCompleted: true,
  });

  const baseline = await costBaseline.getAvgPerMinute(ctx.agentId);
  const currentVelocity = state.spendInWindowCents / WINDOW_SECONDS;

  // Context escalation check: O(n^2) accumulation signature
  const escalationRatio =
    state.previousCallCostCents > 0
      ? state.lastCallCostCents / state.previousCallCostCents
      : 1;

  const shouldOpen =
    (baseline > 0 && currentVelocity > baseline * SPEND_VELOCITY_MULTIPLIER) ||
    escalationRatio > CONTEXT_ESCALATION_RATIO;

  if (shouldOpen) {
    await circuitBreakerStore.open(ctx.agentId, {
      reason:
        escalationRatio > CONTEXT_ESCALATION_RATIO
          ? `context_escalation: ${escalationRatio.toFixed(2)}x cost growth per call`
          : `velocity_anomaly: ${(currentVelocity / baseline).toFixed(1)}x baseline spend rate`,
    });

    await alerting.page({
      severity: "high",
      title: `Circuit opened for agent ${ctx.agentId}`,
      agentId: ctx.agentId,
      featureId: ctx.featureId,
      teamId: ctx.teamId,
      reason: shouldOpen,
    });
  }
}

The escalation ratio check is specifically designed to catch the $47K loop pattern. If each successive call costs 2x the previous call, that is O(n²) context growth. You want to know about this after 3 iterations, not after 264 hours.

Team-Level Budget Allocation

Individual agent budgets are tactical. The organizational governance layer is about how you allocate budget across teams and features, and how you prevent one team’s experimentation from exhausting the budget before another team’s production feature can run.

The allocation model should mirror how headcount or infrastructure budgets work: teams get a period allocation, features within a team draw from that pool, agents within features draw from the feature pool.

async function allocateBudgets(
  month: string,
  totalBudgetCents: number
): Promise<void> {
  const teams = await orgStore.getTeams();

  // First pass: calculate weights based on historical usage and priority tier
  const weights = await Promise.all(
    teams.map(async (team) => {
      const historical = await costStore.getMonthlyAvg(team.id, 3); // 3-month avg
      const tierMultiplier = TIER_MULTIPLIERS[team.priorityTier]; // production > staging > experimental
      return { teamId: team.id, weight: historical * tierMultiplier };
    })
  );

  const totalWeight = weights.reduce((sum, w) => sum + w.weight, 0);

  // Second pass: assign allocations with minimums and caps
  for (const { teamId, weight } of weights) {
    const proportional = (weight / totalWeight) * totalBudgetCents;
    const allocated = Math.min(
      Math.max(proportional, MINIMUM_TEAM_BUDGET_CENTS),
      MAXIMUM_TEAM_BUDGET_CENTS
    );

    await budgetStore.setTeamAllocation(teamId, month, allocated);
  }
}

The priority tier multiplier is doing important organizational work: it ensures that production features with paying customers have a higher claim on the budget than experimental agents running in staging. This is the equivalent of giving production infrastructure a higher cost priority than dev environments.

Model Routing for Cost Tiers

Not every call needs the most expensive model. One of the highest-leverage cost controls is routing calls to the cheapest model that can handle the task.

type ModelTier = "economy" | "standard" | "premium";

interface RoutingPolicy {
  featureId: string;
  defaultTier: ModelTier;
  budgetAwareRouting: boolean;
  degradeToEconomyAt: number; // percent of period budget consumed
}

const MODEL_BY_TIER: Record<ModelTier, string> = {
  economy: "gpt-4o-mini",
  standard: "gpt-4o",
  premium: "claude-sonnet-4-5",
};

async function selectModel(
  ctx: LLMCallContext,
  policy: RoutingPolicy
): Promise<string> {
  if (!policy.budgetAwareRouting) {
    return MODEL_BY_TIER[policy.defaultTier];
  }

  const budget = await budgetStore.getPolicy("feature", ctx.featureId);
  if (!budget) return MODEL_BY_TIER[policy.defaultTier];

  const spentPct = (budget.spentCents / budget.limitCents) * 100;

  // Degrade model tier as budget is consumed
  if (spentPct >= policy.degradeToEconomyAt) {
    return MODEL_BY_TIER["economy"];
  }

  if (spentPct >= policy.degradeToEconomyAt * 0.75) {
    // Intermediate degradation: step down one tier
    const currentIndex = ["economy", "standard", "premium"].indexOf(
      policy.defaultTier
    );
    const degradedTier = ["economy", "standard", "premium"][
      Math.max(0, currentIndex - 1)
    ] as ModelTier;
    return MODEL_BY_TIER[degradedTier];
  }

  return MODEL_BY_TIER[policy.defaultTier];
}

This pattern, budget-aware model degradation, is particularly useful for features where quality is nice-to-have but not required for basic functionality. A background enrichment agent that runs summarization can degrade to a cheaper model at 80% budget consumption without meaningfully impacting the product experience.

Cost Attribution

Attribution is how you connect LLM spend back to business context. Without it, you know you spent $12,000 this month but you cannot say which feature drove it, which team owns the cost, or whether the spend correlated with customer value.

The minimum attribution fields on every LLM call:

interface CostEvent {
  timestamp: Date;
  requestId: string;
  agentId: string;
  featureId: string;
  teamId: string;
  model: string;
  provider: string;
  inputTokens: number;
  outputTokens: number;
  costCents: number;
  latencyMs: number;
  // Business context
  customerId?: string;
  sessionId?: string;
  taskType: string;
  // Outcome signals
  cached: boolean;
  circuitBreakerTripped: boolean;
  budgetDegraded: boolean; // was model downgraded due to budget pressure?
}

The budgetDegraded and circuitBreakerTripped fields are critical for operational reviews. They tell you whether your cost controls are engaging in production, and how often. If budgetDegraded is true on 40% of calls for a feature, that feature is systematically underfunded and needs its allocation reviewed.

Stream these events to your data warehouse. Build a materialized view that gives you daily cost per feature, weekly cost per team, and month-to-date burn vs. allocation. The key query you need to answer every Monday morning: which features are on track to exceed their monthly budget at current run rate?

Tradeoffs

Approach	Enforcement Granularity	Latency Overhead	Operational Complexity
Per-call budget check (Redis)	Per single inference call	1-3ms per call	Low: atomic increment in cache
Per-call budget check (Postgres)	Per single inference call	5-15ms per call	Low: simple row lock
Circuit breaker with state machine	Per agent over time window	2-5ms per call	Medium: state transitions, alerting
Team-level allocation only	Monthly period aggregate	Near zero (async)	Low: background accounting
Pre-call token estimation + rejection	Before API call leaves	0-1ms (local calc)	Medium: estimation accuracy varies by model
Model routing by budget tier	Feature level, automatic	3-8ms per call	Medium: routing policy management

The latency numbers matter at scale. If your LLM calls are 150ms P50, a 15ms synchronous Postgres budget check is 10% overhead. At 1,500ms P50 for a complex agent call, 15ms is noise. Calibrate your enforcement mechanism to your call latency profile.

For high-frequency, low-latency calls (embeddings, classification), use Redis with a Lua script for atomic increment. For low-frequency, high-value calls (complex reasoning agents), Postgres with a row-level lock is fine and gives you better durability guarantees.

Production Considerations

Graceful degradation over hard rejection: For customer-facing features, prefer degrading to a cheaper model over returning a hard error when budgets are tight. Reserve hard rejections for background jobs and experimental agents where a degraded experience is worse than no result. The BudgetExhaustedError should be caught at the orchestration layer and translated to a fallback response where appropriate.

Estimation accuracy: Pre-call token estimation is never exact. Input tokens are easy to count accurately (the prompt is fully formed before the call). Output token estimates are inherently approximate. Use a conservative multiplier (1.2x to 1.5x the expected output tokens) for reservation, and always reconcile actual spend after the call completes.

Period boundary behavior: What happens at midnight on the first of the month? Budget resets need to be atomic. If your budget store resets periods in a background job, there is a window where the old period is exhausted and the new period has not been written yet. Handle this by always allowing calls when a budget record does not exist for the current period, but log the event so you can detect missing budget configurations.

Circuit breaker half-open recovery: After opening a circuit, the system needs a path back to normal operation. The half-open state allows one probe call through. If it succeeds within normal cost parameters, the circuit closes. If it trips the anomaly threshold again, it reopens. Do not auto-close circuits without a probe: a circuit that opened due to a context accumulation loop will immediately reopen if you close it without the loop being fixed.

Alert fatigue is a governance failure: If your spend warnings are firing every day because budgets are too tight, engineers will start ignoring them. Set initial limits generously (2x historical average), then tighten as you build confidence in your baseline data. Alerts should signal genuinely anomalous conditions, not routine usage.

---

Cost governance is not a dashboarding problem. Dashboards show you where you already spent money. Governance prevents the spend in the first place.

The $47K loop did not need better observability. It needed a circuit breaker. The Uber scenario did not need a better cost report. It needed per-team budget gates before the spend happened.

The enforcement layer described here is not complex to build. It is a Redis counter, a circuit breaker state machine, a budget allocation table, and middleware in your LLM call path. The engineering effort is measured in days, not weeks. The alternative is learning about runaway costs from an invoice.