LLM Function Calling in Production: Tool Definitions, Execution Sandboxing, and Error Recovery in TypeScript

The demo works in forty-five minutes. You define a function, describe it, hand it to the model, watch it call the right one with reasonable arguments. You feel like the hard part is done.

The hard part is not done.

Production breaks in ways the demo never reaches: a model generates a valid-looking customer_id that belongs to a different user, a retrieval tool called inside an agent loop returns a document containing adversarial instructions, a chain of five tools where tool three fails and the model retries it indefinitely, an open-source model that ignores the JSON Schema and returns arguments as a freeform string. None of these are edge cases. They are the normal failure surface of function calling at production scale.

This article covers the engineering behind robust tool use: schema design that constrains what models can generate, cross-provider normalization including open-source models, process-level execution sandboxing, error recovery that feeds the model useful feedback, and the fundamental tradeoff between wide and narrow tool catalogs.

The Failure Taxonomy

Before covering solutions, it helps to name the failure categories specifically:

Schema failures: The model generates arguments that pass JSON parsing but fail your business rules. Date ranges where start is after end, UUIDs that are syntactically valid but do not exist, enum values the model hallucinated that are not in the schema.

Authorization failures: Tool arguments reference resources the calling user does not own. The model uses a user ID from earlier in the conversation, from a tool result, or from a system prompt injection.

Execution failures: The tool itself fails: timeouts, downstream API errors, database constraint violations. These are recoverable if fed back correctly, catastrophic if not.

Loop failures: The model calls the same tool repeatedly with the same arguments because it cannot get a satisfying result. Without a circuit breaker, this runs until max_tokens or your budget.

Injection failures: A tool result or retrieved document contains adversarial instructions that the model interprets as new directives, causing it to call tools outside the intended scope.

Sandboxing failures: A tool with filesystem or network access executes operations outside its intended boundary. Path traversal, SSRF, privilege escalation in a multi-tenant environment.

Each failure category requires a different control. Input validation alone does not stop injection. A schema with refine does not stop a model from looping. Authorization at execution time does not protect a filesystem tool without path confinement.

Tool Schema Design

Tool schemas are not documentation. They are the instruction the model uses to generate arguments. The quality of the arguments is bounded by the quality of the schema.

The key design principles: constrain the value space to what is valid, not just what is typed; write descriptions from the model’s decision-making perspective, not yours; derive JSON Schema from your application types, not the other way around.

import { z } from "zod";
import zodToJsonSchema from "zod-to-json-schema";

// Define once, derive JSON Schema for the API call
const QueryTransactionsSchema = z.object({
  account_id: z
    .string()
    .uuid()
    .describe("The UUID of the account to query. Must match the authenticated session."),
  period: z.object({
    from: z.string().datetime().describe("ISO 8601 start date, inclusive."),
    to: z.string().datetime().describe("ISO 8601 end date, inclusive."),
  }),
  category: z
    .enum(["income", "expense", "transfer", "refund"])
    .optional()
    .describe("Filter by transaction category. Omit to return all categories."),
  limit: z
    .number()
    .int()
    .min(1)
    .max(50)
    .default(20)
    .describe("Maximum results. Use lower values when the user asks for a summary."),
});

type QueryTransactionsInput = z.infer<typeof QueryTransactionsSchema>;

function buildToolDefinition(
  name: string,
  description: string,
  schema: z.ZodTypeAny
) {
  const jsonSchema = zodToJsonSchema(schema, {
    name,
    $refStrategy: "none",
  });

  // zodToJsonSchema wraps in a definitions envelope; unwrap it
  const parameters =
    (jsonSchema as Record<string, unknown>).definitions?.[name] ?? jsonSchema;

  return { name, description, parameters };
}

const queryTransactionsTool = buildToolDefinition(
  "query_transactions",
  "Retrieve transactions for an account within a date range. " +
    "Use this when the user asks about spending, income, account history, or recent activity. " +
    "Do not use this to modify transactions.",
  QueryTransactionsSchema
);

The tool description field deserves as much care as the schema. The model uses it for tool selection. Describe the use case, not just the function. Include when to use it and, if relevant, when not to.

Cross-Provider Normalization Including Open-Source Models

OpenAI and Anthropic have similar but not identical APIs. Open-source models (Llama 3, Mistral, Qwen) running via Ollama or vLLM add a third tier with less reliable JSON compliance.

The shape differences matter:

Provider	Tool input format	Response shape	Stop reason
OpenAI	tools[].function.parameters	choices[0].message.tool_calls[]	tool_calls
Anthropic	tools[].input_schema	content[] with tool_use blocks	tool_use
Open-source (via Ollama)	Provider-specific or custom prompt	Often freeform JSON in text	stop with embedded JSON

Normalize at the boundary so the rest of your system never touches provider-specific shapes:

interface NormalizedToolCall {
  id: string;
  name: string;
  input: unknown;
}

interface NormalizedResponse {
  toolCalls: NormalizedToolCall[];
  text: string | null;
  done: boolean; // true if the model is not waiting for tool results
}

function normalizeOpenAI(raw: unknown): NormalizedResponse {
  const r = raw as {
    choices: Array<{
      finish_reason: string;
      message: {
        content: string | null;
        tool_calls?: Array<{
          id: string;
          function: { name: string; arguments: string };
        }>;
      };
    }>;
  };
  const msg = r.choices[0].message;
  const toolCalls = (msg.tool_calls ?? []).map((tc) => ({
    id: tc.id,
    name: tc.function.name,
    input: safeParseJson(tc.function.arguments),
  }));
  return {
    toolCalls,
    text: msg.content,
    done: r.choices[0].finish_reason !== "tool_calls",
  };
}

function normalizeAnthropic(raw: unknown): NormalizedResponse {
  const r = raw as {
    stop_reason: string;
    content: Array<{
      type: string;
      id?: string;
      name?: string;
      input?: unknown;
      text?: string;
    }>;
  };
  const toolCalls = r.content
    .filter((b) => b.type === "tool_use")
    .map((b) => ({ id: b.id!, name: b.name!, input: b.input }));
  const textBlock = r.content.find((b) => b.type === "text");
  return {
    toolCalls,
    text: textBlock?.text ?? null,
    done: r.stop_reason !== "tool_use",
  };
}

// Open-source models frequently embed JSON in their text output
// even when instructed to use a specific format. Parse defensively.
function normalizeOpenSource(raw: unknown): NormalizedResponse {
  const r = raw as {
    done: boolean;
    message: { content: string };
  };

  // Attempt to extract a tool_call JSON block from the text
  const content = r.message.content;
  const jsonMatch = content.match(/```json\s*([\s\S]*?)```/);
  if (jsonMatch) {
    const parsed = safeParseJson(jsonMatch[1]);
    if (
      parsed &&
      typeof parsed === "object" &&
      "name" in (parsed as object) &&
      "arguments" in (parsed as object)
    ) {
      const p = parsed as { name: string; arguments: unknown };
      return {
        toolCalls: [{ id: crypto.randomUUID(), name: p.name, input: p.arguments }],
        text: null,
        done: false, // assume more turns needed
      };
    }
  }

  // No structured call found: treat as text response
  return { toolCalls: [], text: content, done: r.done };
}

function safeParseJson(s: string): unknown {
  try {
    return JSON.parse(s);
  } catch {
    return null;
  }
}

The open-source path is the one most implementations skip. If you run Llama 3 or Mistral locally and your normalization layer expects the OpenAI shape, you get silent failures: the model “completes” without calling any tool, and your agent loop terminates thinking the model answered the question.

For open-source models, the more reliable pattern is to use structured output constraints at the inference layer (llama.cpp grammar, vLLM guided_json) rather than relying on the model to format its own tool calls. When you control the inference runtime, force JSON conformance at the decode level.

Execution Sandboxing

There are two layers of sandboxing that most implementations conflate: argument validation (does the input say to go somewhere it should not?) and execution isolation (even with valid arguments, can this tool affect things outside its intended scope?).

Argument validation is necessary but not sufficient. A path that passes regex validation can still escape a sandbox if you resolve it after receiving it. An argument that looks like a valid HTTP URL can be an SSRF vector if you do not check the resolved host. An argument that conforms to the schema can still carry a prompt injection payload if the value ends up re-entering the context as executable instructions.

Execution isolation is the second layer:

import path from "path";
import { promises as fs } from "fs";

interface FileSystemSandbox {
  workspaceRoot: string;
  maxFileSizeBytes: number;
  allowedExtensions: string[];
  timeoutMs: number;
}

async function sandboxedReadFile(
  rawPath: string,
  sandbox: FileSystemSandbox
): Promise<{ content: string } | { error: string }> {
  // 1. Resolve to an absolute path (catches encoded traversal sequences)
  const resolved = path.resolve(sandbox.workspaceRoot, rawPath);

  // 2. Verify the resolved path is still inside the workspace root
  // NOTE: check resolved path, not rawPath. Encoding tricks bypass rawPath checks.
  if (!resolved.startsWith(sandbox.workspaceRoot + path.sep) &&
      resolved !== sandbox.workspaceRoot) {
    return { error: `Path escapes sandbox: ${rawPath}` };
  }

  // 3. Check extension allowlist before touching the filesystem
  const ext = path.extname(resolved).toLowerCase();
  if (!sandbox.allowedExtensions.includes(ext)) {
    return { error: `File type not allowed: ${ext}` };
  }

  // 4. Check file size before reading
  let stat: Awaited<ReturnType<typeof fs.stat>>;
  try {
    stat = await withTimeout(
      () => fs.stat(resolved),
      sandbox.timeoutMs,
      "stat timed out"
    );
  } catch (err) {
    return { error: err instanceof Error ? err.message : "stat failed" };
  }

  if (stat.size > sandbox.maxFileSizeBytes) {
    return {
      error: `File too large: ${stat.size} bytes (limit: ${sandbox.maxFileSizeBytes})`,
    };
  }

  // 5. Read with timeout
  try {
    const content = await withTimeout(
      () => fs.readFile(resolved, "utf8"),
      sandbox.timeoutMs,
      "read timed out"
    );
    return { content };
  } catch (err) {
    return { error: err instanceof Error ? err.message : "read failed" };
  }
}

async function withTimeout<T>(fn: () => Promise<T>, ms: number, msg: string): Promise<T> {
  return Promise.race([
    fn(),
    new Promise<never>((_, reject) => setTimeout(() => reject(new Error(msg)), ms)),
  ]);
}

For tools that make network requests, the analogous control is a host allowlist applied at request time, not schema validation time:

import { URL } from "url";

function assertAllowedHost(rawUrl: string, allowedHosts: string[]): void {
  let parsed: URL;
  try {
    parsed = new URL(rawUrl);
  } catch {
    throw new Error(`Invalid URL: ${rawUrl}`);
  }

  const hostname = parsed.hostname.toLowerCase();

  // Reject private/loopback ranges explicitly
  const blocked = ["localhost", "127.0.0.1", "0.0.0.0", "::1"];
  if (blocked.includes(hostname)) {
    throw new Error(`Blocked host: ${hostname}`);
  }

  // Also block RFC 1918 patterns if you are in a multi-tenant environment
  const privateRanges = /^(10\.|172\.(1[6-9]|2\d|3[01])\.|192\.168\.)/;
  if (privateRanges.test(hostname)) {
    throw new Error(`Private network host blocked: ${hostname}`);
  }

  if (!allowedHosts.includes(hostname)) {
    throw new Error(`Host not in allowlist: ${hostname}`);
  }
}

The post-resolution path check and the SSRF blocklist are both cases where schema-level validation does not help. The schema knows what the argument says. The sandbox knows where execution would actually go.

Error Recovery: Feeding the Loop

When a tool fails, you have two options: throw an exception that aborts the agent loop, or return a structured error that the model can reason about. The second option is almost always correct.

import Anthropic from "@anthropic-ai/sdk";

type ToolResult = Anthropic.ToolResultBlockParam;

interface ToolExecutionOutcome {
  success: boolean;
  data?: unknown;
  error?: string;
}

async function runToolCallLoop(
  userMessage: string,
  tools: Anthropic.Tool[],
  dispatch: (name: string, input: unknown) => Promise<ToolExecutionOutcome>,
  maxTurns = 12
): Promise<string> {
  const client = new Anthropic();
  const messages: Anthropic.MessageParam[] = [
    { role: "user", content: userMessage },
  ];

  for (let turn = 0; turn < maxTurns; turn++) {
    const response = await client.messages.create({
      model: "claude-opus-4-5",
      max_tokens: 4096,
      tools,
      messages,
    });

    if (response.stop_reason === "end_turn") {
      const textBlock = response.content.find((b) => b.type === "text");
      return textBlock?.type === "text" ? textBlock.text : "";
    }

    if (response.stop_reason !== "tool_use") break;

    const toolUseCalls = response.content.filter(
      (b): b is Anthropic.ToolUseBlock => b.type === "tool_use"
    );

    // Run in parallel; each one returns a tool_result, never throws
    const toolResults: ToolResult[] = await Promise.all(
      toolUseCalls.map(async (call): Promise<ToolResult> => {
        const outcome = await safeDispatch(dispatch, call.name, call.input);
        return {
          type: "tool_result",
          tool_use_id: call.id,
          // Feed error text back as content, not as a thrown exception
          content: outcome.success
            ? JSON.stringify(outcome.data)
            : `Error: ${outcome.error ?? "unknown failure"}`,
          is_error: !outcome.success,
        };
      })
    );

    messages.push({ role: "assistant", content: response.content });
    messages.push({ role: "user", content: toolResults });
  }

  throw new Error(`Agent loop did not complete within ${maxTurns} turns`);
}

// Wraps dispatch so that an unexpected throw becomes a structured failure
async function safeDispatch(
  dispatch: (name: string, input: unknown) => Promise<ToolExecutionOutcome>,
  name: string,
  input: unknown
): Promise<ToolExecutionOutcome> {
  try {
    return await dispatch(name, input);
  } catch (err) {
    return {
      success: false,
      error: err instanceof Error ? err.message : "Unexpected execution error",
    };
  }
}

When a tool returns is_error: true with a descriptive message, a capable model will adjust: try different arguments, ask the user for clarification, or acknowledge that the operation is not possible. When you throw, the loop aborts and the user sees an error with no context about what was attempted or what went wrong.

The maxTurns guard is not a fallback. It is a required control. Set it based on your task complexity, not the theoretical maximum.

Loop Detection and Retry Semantics

Not all retries are the same. A transient network failure warrants a transparent retry. A tool that keeps failing because the model generates wrong arguments needs a different response: feed the error back, not retry the same call.

interface ToolCallRecord {
  name: string;
  inputFingerprint: string;
}

function fingerprint(input: unknown): string {
  return JSON.stringify(input); // stable serialization for comparison
}

function isRepeating(history: ToolCallRecord[], windowSize = 4): boolean {
  if (history.length < windowSize) return false;
  const recent = history.slice(-windowSize);
  const unique = new Set(recent.map((r) => `${r.name}::${r.inputFingerprint}`));
  // Fewer than half distinct calls in the window: treat as a loop
  return unique.size < Math.ceil(windowSize / 2);
}

async function retryTransient<T>(
  fn: () => Promise<T>,
  maxAttempts = 3,
  baseDelayMs = 500
): Promise<T> {
  let lastError: unknown;
  for (let i = 0; i < maxAttempts; i++) {
    try {
      return await fn();
    } catch (err) {
      lastError = err;
      const msg = err instanceof Error ? err.message : "";
      const isTransient =
        msg.includes("timeout") ||
        msg.includes("rate limit") ||
        msg.includes("503") ||
        msg.includes("529");
      if (!isTransient || i === maxAttempts - 1) throw err;
      await new Promise((r) =>
        setTimeout(r, baseDelayMs * Math.pow(2, i))
      );
    }
  }
  throw lastError;
}

The loop detection window size is a parameter you tune per agent. A research agent that legitimately queries multiple sources in sequence looks like a loop if your window is too narrow. An agent that calls the same failing database query four times is definitely looping. Start with a window of four and adjust based on observed turn distributions in production.

The Tool Catalog Tradeoff

This is the architectural question most implementations defer until it hurts: how many tools should the model have access to at once?

More tools means more capability. It also means:

• More input tokens per request (each tool schema adds to the prompt)
• Worse tool selection accuracy as the catalog grows
• A larger attack surface for prompt injection (more tools, more potential actions)
• Harder to reason about what the model might do

Fewer tools means better selection accuracy, lower cost, smaller attack surface, and more predictable behavior. It also means you need a routing layer to determine which tools to include per request.

The tradeoff structured as a decision:

Catalog size	Selection accuracy	Token overhead	Attack surface	Appropriate for
1-5 tools	High	Low	Narrow	Focused agents, single-domain tasks
6-15 tools	Good	Medium	Manageable	Multi-domain assistants with clear task categories
16-40 tools	Degrades	High	Significant	Requires embedding-based pre-filtering
40+ tools	Poor without filtering	Very high	Large	Requires retrieval-based tool selection, not direct inclusion

For large catalogs, the production pattern is two-stage:

interface ToolMetadata {
  name: string;
  description: string;
  embedding: number[]; // precomputed embedding of name + description
  definition: object; // full JSON Schema tool definition
}

// At request time: retrieve the top-K most relevant tools
// using embedding similarity against the user's message
async function selectRelevantTools(
  userMessage: string,
  catalog: ToolMetadata[],
  embedFn: (text: string) => Promise<number[]>,
  topK = 10
): Promise<object[]> {
  const queryEmbedding = await embedFn(userMessage);

  const scored = catalog.map((tool) => ({
    tool,
    score: cosineSimilarity(queryEmbedding, tool.embedding),
  }));

  scored.sort((a, b) => b.score - a.score);

  return scored.slice(0, topK).map((s) => s.tool.definition);
}

function cosineSimilarity(a: number[], b: number[]): number {
  let dot = 0;
  let normA = 0;
  let normB = 0;
  for (let i = 0; i < a.length; i++) {
    dot += a[i] * b[i];
    normA += a[i] * a[i];
    normB += b[i] * b[i];
  }
  return dot / (Math.sqrt(normA) * Math.sqrt(normB));
}

This is the same retrieval pattern you would use for RAG, applied to tool selection. The cost is one embedding call per request. The benefit is that you send the model ten focused tools instead of fifty, with meaningfully better selection accuracy and lower token overhead.

Observability for Tool Call Chains

Tool calls in an agent loop form a dependency chain. Standard request metrics do not capture this: you need per-call instrumentation that retains the chain structure and allows you to reconstruct what happened across a full agent run.

The minimum useful event per tool call:

interface ToolCallEvent {
  agent_run_id: string;
  session_id: string;
  turn_index: number;
  call_index: number;
  tool_name: string;
  // Do not log full input: may contain PII or sensitive data
  input_fingerprint: string;
  started_at: number;
  duration_ms: number;
  success: boolean;
  error_category:
    | "schema"
    | "auth"
    | "sandbox"
    | "transient"
    | "business_rule"
    | "loop_detected"
    | null;
  output_bytes: number | null;
}

Aggregate metrics that surface real problems:

• Schema error rate per tool: an upward trend means your schema or prompt changed in a way that breaks argument generation
• Auth denial rate: a non-zero baseline means either your authorization is too strict or a prompt injection is attempting out-of-scope tool calls
• Loop detection triggers: any non-zero value in production needs investigation; find the prompt or tool combination that triggers it
• P95 tool latency per tool: one slow tool dominates agent turn time
• Output bytes P95 per tool: large tool outputs inflate context window usage and cost; set a max and truncate before returning to the model

Production Considerations

Version your tool schemas. The schema is an API contract with the model. Changing a parameter name, removing an enum value, or making an optional field required is a breaking change for agents already in flight. Use additive changes where possible. When you must break compatibility, version the tool name explicitly (query_transactions_v2) and retire the old version with a grace period.

Set per-tool rate limits at the executor level. A model in a reasoning loop will call an expensive tool repeatedly before your turn limit fires. Apply per-tool call caps within a single turn (not just total turns). A web search tool might allow three calls per turn. A payment-processing tool might allow one.

Idempotency keys for mutation tools. Agent loops can replay. If a turn fails after a mutation tool executed but before the response was recorded, the next attempt will call the tool again. Pass an idempotency key derived from the session ID and turn index into every mutation tool. Make the tool’s downstream operation idempotent on that key.

Sanitize tool results before they re-enter context. Treat every tool result as untrusted input. Strip or escape content that could be interpreted as instructions. For tools that retrieve documents or external data, apply a maximum size cap before the result goes back into the model’s context, or run a summarization step first. The risk is that a retrieved document contains adversarial text that hijacks subsequent tool calls.

Test the failure paths, not just the happy path. Your test suite should include: missing required fields, type mismatches, out-of-range values, path traversal attempts on filesystem tools, SSRF attempts on network tools, prompt injection payloads in tool results, and scenarios where a tool fails mid-chain. These are not edge cases; they are the normal attack surface.

Function calling is the interface between language models and your infrastructure. That interface has a security perimeter, an authorization boundary, a cost model, and a set of failure modes that are entirely predictable once you have seen them. The afternoon prototype leaves all of that implicit. The production system makes it explicit, enforces it consistently, and measures it continuously.

The gap between demo and production is not complexity. It is discipline about where trust lives: trust the model to select tools and generate plausible arguments, and nothing else. Everything downstream of that decision is yours to control.