Multi-Agent Orchestration: Patterns for Coordinating LLM Agents in Production

Single-agent systems have a natural ceiling. One LLM with a set of tools works well for focused tasks: answer a support ticket, summarize a document, write a database query. But when the task requires true parallelism, specialized reasoning across domains, or checks and balances to reduce hallucination, a single agent becomes a bottleneck or a liability.

Multi-agent systems solve some of these problems. They also introduce new ones that do not show up until you are in production.

This article covers the four core orchestration patterns, how to implement them in TypeScript, how to manage state across agents, and the failure modes that emerge specifically from agent interaction. It also covers the question that gets skipped most often: when is the complexity worth it, and when should you just give your single agent better tools?

The Case Against Multi-Agent (Start Here)

Before adding orchestration complexity, rule out the simpler alternatives.

A single agent fails for predictable reasons: the context window fills up, the model loses track of the original goal over many steps, the task requires knowledge from two domains that one prompt cannot combine well, or throughput is limited by a single sequential execution path.

Before splitting into multiple agents, check:

• Context compression: Can you summarize intermediate results to keep context manageable?
• Better tool design: Can a single well-scoped tool replace a separate agent?
• Parallel tool calls: Most APIs support calling multiple tools simultaneously in one LLM turn. This handles many parallelism needs without a second agent.
• Structured outputs with post-processing: Sometimes the “second agent for verification” is just a deterministic check on structured JSON.

Multi-agent is the right answer when the task genuinely requires independent reasoning from parallel workers, when you need fault isolation (one agent failing should not bring down the whole task), or when different subtasks benefit from different system prompts, model sizes, or tool sets.

The Four Patterns

1. Supervisor

One orchestrator agent receives the task, decomposes it, delegates to specialist sub-agents, collects results, and synthesizes the final output.

interface SubAgent {
  name: string;
  description: string;
  systemPrompt: string;
  tools: Tool[];
  model: string;
}

interface SupervisorState {
  originalTask: string;
  decomposition: SubTask[];
  results: Map<string, SubTaskResult>;
  finalAnswer?: string;
}

interface SubTask {
  id: string;
  description: string;
  assignedAgent: string;
  dependsOn: string[]; // IDs of subtasks that must complete first
}

async function runSupervisor(
  task: string,
  subAgents: SubAgent[]
): Promise<string> {
  // Step 1: Orchestrator decomposes the task
  const decomposition = await orchestratorLLM({
    system: `You are a task orchestrator. Given a complex task, decompose it into
      subtasks and assign each to the appropriate specialist.
      Available specialists: ${subAgents.map(a => `${a.name}: ${a.description}`).join(", ")}
      Respond with JSON matching the SubTask[] schema.`,
    user: task,
    schema: SubTaskArraySchema,
  });

  const state: SupervisorState = {
    originalTask: task,
    decomposition,
    results: new Map(),
  };

  // Step 2: Execute subtasks respecting dependencies
  await executeWithDependencies(state, subAgents);

  // Step 3: Synthesize
  const synthesis = await orchestratorLLM({
    system: "Synthesize the subtask results into a final answer.",
    user: JSON.stringify({
      originalTask: state.originalTask,
      results: Object.fromEntries(state.results),
    }),
  });

  return synthesis;
}

async function executeWithDependencies(
  state: SupervisorState,
  subAgents: SubAgent[]
): Promise<void> {
  const completed = new Set<string>();
  const pending = [...state.decomposition];

  while (pending.length > 0) {
    // Find tasks whose dependencies are satisfied
    const ready = pending.filter(t =>
      t.dependsOn.every(dep => completed.has(dep))
    );

    if (ready.length === 0) {
      throw new Error("Circular dependency or unresolvable dependency chain");
    }

    // Run ready tasks in parallel
    await Promise.all(
      ready.map(async (subtask) => {
        const agent = subAgents.find(a => a.name === subtask.assignedAgent);
        if (!agent) throw new Error(`Unknown agent: ${subtask.assignedAgent}`);

        const context = buildContext(subtask, state.results);
        const result = await runSubAgent(agent, subtask.description, context);

        state.results.set(subtask.id, result);
        completed.add(subtask.id);
      })
    );

    // Remove completed tasks from pending
    pending.splice(0, pending.length, ...pending.filter(t => !completed.has(t.id)));
  }
}

The supervisor pattern is the most flexible but also the most fragile. The orchestrator’s decomposition quality determines everything downstream. If it assigns the wrong agent to a subtask or creates a dependency cycle, the whole run fails. The orchestrator itself is a single point of failure.

When to use it: Tasks with genuine subtask diversity (different domains, different tools, different reasoning styles required). Research and synthesis workflows. Customer support routing where different request types need specialist handling.

2. Pipeline

Agents run in sequence, each receiving the previous agent’s output as input. No central orchestrator: each agent is a stage in a predefined flow.

interface PipelineStage {
  name: string;
  agent: SubAgent;
  transform?: (input: unknown, output: unknown) => unknown; // Optional output shaping
}

interface PipelineState {
  input: unknown;
  stages: PipelineStage[];
  stageOutputs: Map<string, unknown>;
  currentStage: number;
}

async function runPipeline(
  input: unknown,
  stages: PipelineStage[]
): Promise<unknown> {
  let current = input;

  for (const stage of stages) {
    const result = await runSubAgent(
      stage.agent,
      typeof current === "string" ? current : JSON.stringify(current)
    );

    const output = stage.transform ? stage.transform(current, result) : result;

    // Persist stage output for debugging and recovery
    await persistStageOutput(stage.name, output);

    current = output;
  }

  return current;
}

Pipelines are deterministic, easy to monitor, and easy to resume from a checkpoint. If stage 3 fails, you can reload the stage 2 output and retry without rerunning stages 0 and 1. This is a significant operational advantage over the supervisor pattern.

The tradeoff: no flexibility. The pipeline cannot adapt its structure based on intermediate results. If the task requires branching, you need conditional stage execution or a supervisor.

interface ConditionalPipelineStage extends PipelineStage {
  condition?: (previousOutput: unknown) => boolean;
}

async function runConditionalPipeline(
  input: unknown,
  stages: ConditionalPipelineStage[]
): Promise<unknown> {
  let current = input;

  for (const stage of stages) {
    if (stage.condition && !stage.condition(current)) {
      // Skip this stage
      continue;
    }

    current = await runSubAgentWithRetry(stage.agent, current);
    await checkpointStage(stage.name, current);
  }

  return current;
}

When to use it: Document processing workflows (extract, enrich, validate, format). ETL-style AI pipelines. Any task with a fixed, known sequence of transformations where observability and recoverability matter.

3. Debate / Consensus

Multiple agents independently produce answers to the same question, then either vote on the best answer or critique each other’s answers in rounds until consensus forms.

This pattern is specifically for reducing hallucination in high-stakes decisions. It does not make sense for creative tasks where variation is the point.

interface DebateConfig {
  question: string;
  agents: SubAgent[];
  rounds: number;
  consensusThreshold: number; // 0-1, fraction of agents that must agree
}

async function runDebate(config: DebateConfig): Promise<DebateResult> {
  // Round 1: Independent answers
  const initialAnswers = await Promise.all(
    config.agents.map(agent =>
      runSubAgent(agent, config.question)
    )
  );

  if (config.rounds === 1) {
    return aggregateVotes(initialAnswers);
  }

  // Subsequent rounds: agents see each other's answers and can revise
  let currentAnswers = initialAnswers;

  for (let round = 2; round <= config.rounds; round++) {
    const context = buildDebateContext(currentAnswers);

    currentAnswers = await Promise.all(
      config.agents.map((agent, idx) =>
        runSubAgent(agent, `
          Question: ${config.question}

          Other agents answered:
          ${context.filter((_, i) => i !== idx).join("\n---\n")}

          Review these answers. You may maintain your position or revise it.
          Explain your reasoning briefly.
        `)
      )
    );

    const consensus = checkConsensus(currentAnswers, config.consensusThreshold);
    if (consensus.reached) {
      return { answer: consensus.answer, rounds: round, confidence: "high" };
    }
  }

  // No consensus: return majority answer with low confidence flag
  return { ...aggregateVotes(currentAnswers), confidence: "low" };
}

function aggregateVotes(answers: string[]): DebateResult {
  // Extract structured answers (assumes agents output JSON with an "answer" field)
  const parsed = answers.map(a => {
    try { return JSON.parse(a); }
    catch { return { answer: a }; }
  });

  // Find the most common answer
  const counts = new Map<string, number>();
  for (const { answer } of parsed) {
    counts.set(answer, (counts.get(answer) ?? 0) + 1);
  }

  const [topAnswer, topCount] = [...counts.entries()]
    .sort(([, a], [, b]) => b - a)[0];

  return {
    answer: topAnswer,
    confidence: topCount / answers.length >= 0.67 ? "high" : "medium",
    rounds: 1,
  };
}

The cost of this pattern is roughly N times the token cost of a single answer, per round. For three agents over two rounds, you spend 6x the tokens. Use it when the stakes justify that cost: medical information summarization, financial analysis, legal document interpretation.

When to use it: High-stakes factual questions where hallucination is costly. Situations where you want to surface disagreement rather than a single answer. Classification tasks where inter-rater reliability matters.

4. Map-Reduce

A coordinator splits large input into chunks, fans out to worker agents in parallel (map phase), then aggregates the results (reduce phase).

interface MapReduceConfig<TChunk, TResult, TFinal> {
  input: TChunk[];
  mapAgent: SubAgent;
  reduceAgent: SubAgent;
  chunkToPrompt: (chunk: TChunk, index: number, total: number) => string;
  concurrency: number; // Max parallel map workers
}

async function runMapReduce<TChunk, TResult, TFinal>(
  config: MapReduceConfig<TChunk, TResult, TFinal>
): Promise<TFinal> {
  // Map phase with concurrency control
  const mapped = await mapWithConcurrency(
    config.input,
    config.concurrency,
    async (chunk, index) => {
      const prompt = config.chunkToPrompt(chunk, index, config.input.length);
      return runSubAgent(config.mapAgent, prompt);
    }
  );

  // Reduce phase
  const reduced = await runSubAgent(
    config.reduceAgent,
    `Aggregate the following ${mapped.length} results:\n\n${mapped.join("\n---\n")}`
  );

  return JSON.parse(reduced) as TFinal;
}

async function mapWithConcurrency<T, R>(
  items: T[],
  concurrency: number,
  fn: (item: T, index: number) => Promise<R>
): Promise<R[]> {
  const results: R[] = new Array(items.length);
  let index = 0;

  async function worker(): Promise<void> {
    while (index < items.length) {
      const current = index++;
      results[current] = await fn(items[current], current);
    }
  }

  await Promise.all(
    Array.from({ length: Math.min(concurrency, items.length) }, worker)
  );

  return results;
}

The reduce phase is a bottleneck. If you have 100 map results and need to aggregate them, the reduce agent gets a massive prompt. For very large fan-outs, use a hierarchical reduce: aggregate in batches, then aggregate the batch results.

async function hierarchicalReduce(
  results: string[],
  batchSize: number,
  reduceAgent: SubAgent
): Promise<string> {
  if (results.length <= batchSize) {
    return runSubAgent(reduceAgent, results.join("\n---\n"));
  }

  // Reduce in batches
  const batches = chunkArray(results, batchSize);
  const batchResults = await Promise.all(
    batches.map(batch => runSubAgent(reduceAgent, batch.join("\n---\n")))
  );

  // Recursively reduce batch results
  return hierarchicalReduce(batchResults, batchSize, reduceAgent);
}

When to use it: Processing documents too large for one context window. Parallel analysis of a dataset. Summarizing a long corpus by chunks. Any task where the input can be cleanly partitioned and independently processed.

Pattern Comparison

Pattern	Flexibility	Recoverability	Cost	Complexity	Best For
Supervisor	High	Low	Medium	High	Diverse subtasks, unknown decomposition
Pipeline	Low	High	Low	Low	Fixed sequence, ETL-style workflows
Debate	None	Medium	High	Medium	High-stakes factual decisions
Map-Reduce	Medium	High	Medium	Medium	Large input parallelism

State Management Between Agents

The hardest part of multi-agent systems is not the individual agent logic. It is state: what each agent knows, what it can write, and how you recover when something fails mid-run.

Shared state store

Every non-trivial multi-agent system needs a shared state store outside of any individual agent’s context. In-process objects do not survive restarts and do not work across distributed workers.

interface AgentRunState {
  runId: string;
  pattern: "supervisor" | "pipeline" | "debate" | "map-reduce";
  status: "running" | "completed" | "failed" | "paused";
  startedAt: string;
  updatedAt: string;
  input: unknown;
  agentOutputs: Record<string, unknown>; // keyed by agent name + stage
  error?: string;
  finalOutput?: unknown;
}

class RunStateStore {
  constructor(private kv: KVStore) {}

  async save(state: AgentRunState): Promise<void> {
    await this.kv.set(`run:${state.runId}`, JSON.stringify(state));
  }

  async load(runId: string): Promise<AgentRunState | null> {
    const raw = await this.kv.get(`run:${runId}`);
    return raw ? JSON.parse(raw) : null;
  }

  async updateOutput(
    runId: string,
    key: string,
    output: unknown
  ): Promise<void> {
    const state = await this.load(runId);
    if (!state) throw new Error(`Run ${runId} not found`);

    state.agentOutputs[key] = output;
    state.updatedAt = new Date().toISOString();
    await this.save(state);
  }
}

Write state after each agent completes, not at the end. That way a failure in agent 5 of 8 does not require rerunning agents 1 through 4.

Avoiding shared state mutation

The most common source of bugs in multi-agent systems is agents reading and writing the same state concurrently. The supervisor pattern is particularly prone to this: if two sub-agents both update a shared result object, one update silently overwrites the other.

Design around this by giving each agent a dedicated output namespace and merging at the supervisor level:

// Bad: agents write to shared state directly
state.results["analysis"] = await agent1.run(input);
state.results["summary"] = await agent2.run(input); // may race with agent1

// Good: agents write to isolated keys, supervisor merges
const [analysis, summary] = await Promise.all([
  agent1.run(input),
  agent2.run(input),
]);
state.results = { analysis, summary }; // atomic merge after both complete

Error Handling and Recovery

Single-agent error handling is hard. Multi-agent error handling requires decisions about what to do when one agent in a larger run fails.

type AgentFailurePolicy =
  | "fail-fast"     // Abort entire run on any agent failure
  | "best-effort"   // Continue with whatever succeeds, note failures
  | "retry-once"    // Retry the failed agent once, then fail-fast
  | "fallback";     // Use a fallback agent or default output

interface ResilientAgentRunner {
  policy: AgentFailurePolicy;
  fallback?: SubAgent;
  maxRetries?: number;
}

async function runWithPolicy(
  agent: SubAgent,
  input: string,
  policy: AgentFailurePolicy,
  options: { fallback?: SubAgent; maxRetries?: number } = {}
): Promise<{ output: string; usedFallback: boolean }> {
  const maxRetries = options.maxRetries ?? 1;

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      const output = await runSubAgent(agent, input);
      return { output, usedFallback: false };
    } catch (err) {
      if (attempt === maxRetries) {
        if (policy === "fail-fast") throw err;

        if (policy === "best-effort") {
          return { output: `[Agent ${agent.name} failed: ${String(err)}]`, usedFallback: false };
        }

        if (policy === "fallback" && options.fallback) {
          const fallbackOutput = await runSubAgent(options.fallback, input);
          return { output: fallbackOutput, usedFallback: true };
        }

        throw err;
      }
      // Will retry
      await sleep(1000 * (attempt + 1)); // simple backoff
    }
  }

  throw new Error("Unreachable");
}

For the map-reduce pattern, best-effort is often the right policy for the map phase: if 2 out of 50 chunks fail to process, you can still produce a useful result from 48. For the reduce phase, fail-fast is usually correct because a partial reduce produces a misleading result.

Tool-Use Coordination

When agents in the same system share tools, you need to think about resource contention and idempotency.

interface SharedTool extends Tool {
  concurrencyLimit: number; // Max simultaneous calls
  idempotencyKey?: (params: unknown) => string; // For deduplication
}

class ToolRegistry {
  private semaphores = new Map<string, Semaphore>();

  register(tool: SharedTool): void {
    this.semaphores.set(tool.name, new Semaphore(tool.concurrencyLimit));
  }

  async execute(toolName: string, params: unknown): Promise<unknown> {
    const semaphore = this.semaphores.get(toolName);
    if (!semaphore) throw new Error(`Tool ${toolName} not registered`);

    return semaphore.withLock(() => {
      const tool = this.tools.get(toolName)!;
      return tool.execute(params);
    });
  }
}

Rate-limited external APIs (search, scraping, database writes) need a centralized rate limiter that all agents share. Without it, five parallel agents will each try to hit your rate limit independently, and all but one will fail with 429s.

For write tools, idempotency is critical. If a map-reduce worker retries a write because it got a timeout, it should not create a duplicate record. Design write tools to accept an idempotency key and check for existing records before writing.

Failure Modes Specific to Multi-Agent Systems

Beyond the failures that affect single agents, these emerge from coordination itself.

Cascading failure: one sub-agent returns a malformed result, the supervisor passes it to the next agent without validation, and the error propagates through the system in a way that is hard to trace. Fix: validate each agent’s output at the handoff point, not just at the end.

Deadlock in dependency graphs: a supervisor decomposes a task into subtasks where A depends on B and B depends on A. The system hangs forever. Fix: detect cycles before starting execution. Use topological sort on the dependency graph and fail fast on cycles.

function detectCycles(tasks: SubTask[]): string[] {
  const visiting = new Set<string>();
  const visited = new Set<string>();
  const cycles: string[] = [];

  function dfs(taskId: string, path: string[]): void {
    if (visiting.has(taskId)) {
      cycles.push([...path, taskId].join(" -> "));
      return;
    }
    if (visited.has(taskId)) return;

    visiting.add(taskId);
    const task = tasks.find(t => t.id === taskId);
    for (const dep of task?.dependsOn ?? []) {
      dfs(dep, [...path, taskId]);
    }
    visiting.delete(taskId);
    visited.add(taskId);
  }

  for (const task of tasks) dfs(task.id, []);
  return cycles;
}

Context contamination: in debate patterns, agents reading each other’s answers early can anchor to the first answer rather than reason independently. This defeats the purpose of the debate. Use a two-phase approach: collect independent answers before sharing any of them.

Silent partial completion: a supervisor declares success after 7 of 8 subtasks complete because the 8th timed out and returned a soft error. Without explicit tracking of which subtasks were skipped, the caller believes the task is done. Always track skipped and failed subtasks explicitly in the state.

Orchestrator hallucinating agent capabilities: in supervisor patterns, the orchestrator may assign a subtask to an agent that does not have the right tools for it. The sub-agent then fails, often with a confusing error. Fix: include tool names in agent descriptions, and have the orchestrator validate assignments against a schema before starting execution.

Observability

Multi-agent runs are much harder to debug than single-agent runs because failures may be caused by the interaction between agents, not by any individual agent.

Every agent run should emit a structured trace event:

interface AgentSpan {
  runId: string;
  spanId: string;
  parentSpanId?: string; // Links sub-agent spans to supervisor spans
  agentName: string;
  pattern: string;
  startedAt: string;
  completedAt?: string;
  inputTokens: number;
  outputTokens: number;
  model: string;
  toolCalls: Array<{
    toolName: string;
    success: boolean;
    durationMs: number;
  }>;
  status: "success" | "failure" | "timeout";
  error?: string;
}

The parentSpanId is what makes multi-agent traces useful. You can reconstruct the full call tree from spans alone, see where time was spent, and identify which agent in the chain produced a bad output that corrupted downstream results.

When Single-Agent with Better Tooling Wins

The argument for staying single-agent is operational simplicity: one thing to monitor, one thing to debug, one failure mode. Before shipping a multi-agent system, ask these questions.

Can parallel tool calls replace agent parallelism? Most LLM APIs support calling multiple tools in a single turn. Three read operations that do not depend on each other can run in parallel without a second agent.

Can a structured output replace a verification agent? If the “debate” is to catch hallucinations, sometimes a constrained JSON output schema with a deterministic validator catches errors just as well and costs nothing.

Can a larger context window replace map-reduce? If the whole document fits in 200K tokens, a single Gemini or Claude call may be cheaper and more accurate than splitting, mapping, and reducing across multiple agents.

The honest answer is that most tasks that feel like they need multi-agent are actually single-agent with inadequate tooling or poorly designed prompts. Run the simpler version first. Measure where it fails. Add orchestration only where the failure mode is genuinely architectural.

Production Considerations

For multi-agent systems that run in production:

Set per-agent budgets: each sub-agent should have its own step limit and token limit, independent of the overall run budget. An agent stuck in a loop should not consume the entire run’s budget.

Log at every handoff: log the full input and output at every agent boundary. This is verbose but it is the only way to diagnose interaction failures after the fact.

Implement run-level timeouts: individual agent timeouts are necessary but not sufficient. The overall orchestration needs a hard deadline. A slow sub-agent can hold up a supervisor run indefinitely without one.

Plan for partial results: design the calling interface to return partial results with a status field rather than blocking until full completion. Long-running orchestrations need to be observable and interruptible.

Version your prompts: when a multi-agent system breaks after a model update, you need to know which agent’s behavior changed. Treat system prompts as versioned artifacts and log which version ran with each span.

Closing

Multi-agent systems are not complex by default. A two-stage pipeline is simpler than a well-prompted single agent that tries to do too much in one context. The complexity comes from the coordination layer: dependency management, shared state, partial failure handling, and the emergent bugs that only appear when agents interact.

Start with the simplest pattern that solves the actual problem. For most document-processing and analysis tasks, a linear pipeline with per-stage checkpointing is the right architecture. Supervisor patterns earn their complexity only when the task structure is genuinely dynamic. Debate patterns are for high-stakes factual tasks where you can afford the cost. Map-reduce is for scale.

Whatever pattern you choose, instrument the handoffs more than the individual agents. That is where the interesting failures live.