Building a Tool-Use Layer for LLM Agents: Function Calling, Schema Validation, and Error Recovery

Most function calling implementations start as glue code: a switch statement that dispatches tool names to functions, error handling bolted on after the first production incident, and observability added when something goes silently wrong. This is fine for a prototype. It is not fine once you have multiple agents, a growing tool catalog, and the expectation that tool failures are handled gracefully rather than silently corrupting agent state.

The failure modes that matter in production are not the obvious ones. The model will call a real tool with well-formed arguments that still cause damage: a deletion tool called with a valid-but-wrong ID, a search tool called 40 times in a conversation because the result was ambiguous, a file-write tool called with a path that escapes the intended sandbox. You will not catch these with input validation alone.

This article covers building a tool-use layer as infrastructure, not glue code: a provider abstraction that normalizes OpenAI and Anthropic differences, a schema-first design pipeline, a pre-execution gate for dangerous tools, retry strategies specific to agent loops, and structured observability for tool execution chains.

Provider Abstraction: Normalizing OpenAI and Anthropic

The two dominant APIs are close but not identical. OpenAI uses tools with a function object wrapper. Anthropic uses tools with a flat schema alongside input_schema. The response shapes differ too: OpenAI returns tool_calls on the message, Anthropic returns a content array that mixes text and tool_use blocks. Stop reasons differ: tool_calls vs tool_use.

If you hardcode one provider’s shape, a provider swap becomes a refactor. Instead, normalize at the boundary:

import { z } from "zod";
import zodToJsonSchema from "zod-to-json-schema";

// Provider-agnostic tool definition
interface ToolDefinition<TInput extends z.ZodTypeAny = z.ZodTypeAny> {
  name: string;
  description: string;
  inputSchema: TInput;
  dangerous?: boolean; // requires sandbox gate
  maxCallsPerTurn?: number; // rate-limit per agent turn
}

// Serialized for OpenAI
function toOpenAITool(tool: ToolDefinition): object {
  const jsonSchema = zodToJsonSchema(tool.inputSchema, {
    $refStrategy: "none",
    name: tool.name,
  });
  return {
    type: "function",
    function: {
      name: tool.name,
      description: tool.description,
      parameters: (jsonSchema as Record<string, unknown>).definitions?.[tool.name] ?? jsonSchema,
    },
  };
}

// Serialized for Anthropic
function toAnthropicTool(tool: ToolDefinition): object {
  const jsonSchema = zodToJsonSchema(tool.inputSchema, {
    $refStrategy: "none",
    name: tool.name,
  });
  return {
    name: tool.name,
    description: tool.description,
    input_schema: (jsonSchema as Record<string, unknown>).definitions?.[tool.name] ?? jsonSchema,
  };
}

// Normalized tool call extracted from either provider's response
interface NormalizedToolCall {
  id: string;
  name: string;
  input: unknown;
}

function extractToolCalls(response: unknown, provider: "openai" | "anthropic"): NormalizedToolCall[] {
  if (provider === "openai") {
    const msg = response as { choices: Array<{ message: { tool_calls?: Array<{ id: string; function: { name: string; arguments: string } }> } }> };
    return (msg.choices[0]?.message?.tool_calls ?? []).map((tc) => ({
      id: tc.id,
      name: tc.function.name,
      input: JSON.parse(tc.function.arguments),
    }));
  }
  const msg = response as { content: Array<{ type: string; id?: string; name?: string; input?: unknown }> };
  return msg.content
    .filter((b) => b.type === "tool_use")
    .map((b) => ({ id: b.id!, name: b.name!, input: b.input }));
}

This boundary isolates provider differences to two serialization functions and one extraction function. The rest of your tool-use layer operates on ToolDefinition and NormalizedToolCall regardless of which model is running the agent.

Schema-First Tool Design

The quality of tool arguments the model generates is directly proportional to the quality of the schema and description you provide. A query: string parameter gets a free-form string. A query: string with .min(3).max(200) and a description that says “the user’s search terms, plain English, no Boolean operators” gets meaningfully better arguments at inference time because the constraints are visible in the schema the model receives.

Design schemas to express what is valid, not just what the type system requires:

import { z } from "zod";

const DeleteFileSchema = z.object({
  path: z
    .string()
    .regex(/^[a-zA-Z0-9_\-./]+$/, "path must contain only safe characters")
    .refine((p) => !p.includes(".."), "path traversal not allowed")
    .describe("Relative path within the workspace. No absolute paths. No .. traversal."),
  confirm: z
    .literal(true)
    .describe("Must be explicitly true. The model must confirm intent before deletion."),
});

const SearchCodeSchema = z.object({
  pattern: z
    .string()
    .min(2)
    .max(200)
    .describe("The search pattern. Supports regex syntax."),
  file_glob: z
    .string()
    .default("**/*")
    .describe("Glob pattern to restrict which files are searched. Default: all files."),
  max_results: z
    .number()
    .int()
    .min(1)
    .max(50)
    .default(20)
    .describe("Maximum results to return. Lower is faster for broad patterns."),
  case_sensitive: z.boolean().default(false),
});

The confirm: z.literal(true) pattern on DeleteFileSchema is a forcing function. The model cannot call the delete tool without explicitly setting confirm: true in its arguments, which means the system prompt must tell it when that is appropriate. It does not prevent misuse, but it makes accidental deletion significantly less likely and provides a clear signal in logs when a deletion is triggered.

Derive your JSON Schema from Zod rather than writing it by hand. The schema the model receives is code. Treat it that way: version it, test it, review changes before deploying.

Pre-Execution Validation Gate

Structural validation (does the input match the schema?) is the minimum. Production tool execution requires a second gate: a per-tool validator that applies business rules and authorization checks before any side effect runs.

interface ToolExecutionContext {
  sessionId: string;
  userId: string;
  workspaceId: string;
  turnIndex: number;
  callCountThisTurn: Map<string, number>;
}

interface ValidationResult {
  allowed: boolean;
  reason?: string;
}

type PreExecutionValidator<T> = (
  args: T,
  ctx: ToolExecutionContext
) => Promise<ValidationResult>;

interface RegisteredTool<TInput extends z.ZodTypeAny = z.ZodTypeAny> {
  definition: ToolDefinition<TInput>;
  handler: (args: z.infer<TInput>, ctx: ToolExecutionContext) => Promise<unknown>;
  preExecute?: PreExecutionValidator<z.infer<TInput>>;
}

async function executeToolCall(
  call: NormalizedToolCall,
  tool: RegisteredTool,
  ctx: ToolExecutionContext
): Promise<{ success: boolean; result: unknown; error?: string }> {
  // 1. Structural validation
  const parsed = tool.definition.inputSchema.safeParse(call.input);
  if (!parsed.success) {
    return {
      success: false,
      result: null,
      error: `Schema validation failed: ${parsed.error.issues
        .map((i) => `${i.path.join(".")}: ${i.message}`)
        .join("; ")}`,
    };
  }

  // 2. Rate limit check
  const maxCalls = tool.definition.maxCallsPerTurn ?? Infinity;
  const callCount = ctx.callCountThisTurn.get(call.name) ?? 0;
  if (callCount >= maxCalls) {
    return {
      success: false,
      result: null,
      error: `Tool ${call.name} has been called ${callCount} times this turn (limit: ${maxCalls})`,
    };
  }

  // 3. Business rule / authorization gate
  if (tool.preExecute) {
    const check = await tool.preExecute(parsed.data, ctx);
    if (!check.allowed) {
      return {
        success: false,
        result: null,
        error: check.reason ?? "Pre-execution check denied",
      };
    }
  }

  // 4. Execute
  ctx.callCountThisTurn.set(call.name, callCount + 1);
  try {
    const result = await tool.handler(parsed.data, ctx);
    return { success: true, result };
  } catch (err) {
    return {
      success: false,
      result: null,
      error: err instanceof Error ? err.message : "Tool execution threw an unexpected error",
    };
  }
}

The preExecute hook is where authorization lives. Do not derive the user’s identity from model arguments. Derive it from ctx, which you populate from your session or token before the agent loop starts. The model can claim any user ID it wants in its arguments. Your context is the authoritative source.

Sandboxing Dangerous Tools

Some tools are inherently risky: file writes, shell execution, database mutations, network requests to external services. These need a sandbox layer that constrains what the tool can affect, regardless of the arguments the model provides.

The sandbox is not about distrusting the model specifically. It is about defense-in-depth. A prompt injection attack, a model hallucination, or a misconfigured system prompt should not be able to cause irreversible damage.

interface SandboxConfig {
  allowedPaths?: string[]; // for filesystem tools: restrict to these prefixes
  allowedHosts?: string[]; // for network tools: allowlist of destinations
  timeoutMs: number; // hard execution timeout
  maxOutputBytes: number; // cap tool output size
}

function createSandboxedFileWriter(config: SandboxConfig) {
  return async (args: { path: string; content: string }, ctx: ToolExecutionContext) => {
    const workspaceRoot = `/workspaces/${ctx.workspaceId}`;
    const resolvedPath = require("path").resolve(workspaceRoot, args.path);

    // Enforce path confinement after resolution (catches encoded traversal)
    if (!resolvedPath.startsWith(workspaceRoot)) {
      throw new Error(`Path ${args.path} resolves outside the workspace boundary`);
    }

    // Cap content size before hitting the filesystem
    const encoded = Buffer.from(args.content, "utf8");
    if (encoded.byteLength > config.maxOutputBytes) {
      throw new Error(
        `Content size ${encoded.byteLength} bytes exceeds limit of ${config.maxOutputBytes} bytes`
      );
    }

    return withTimeout(
      () => fs.writeFile(resolvedPath, args.content, "utf8"),
      config.timeoutMs,
      `write_file timed out after ${config.timeoutMs}ms`
    );
  };
}

async function withTimeout<T>(
  fn: () => Promise<T>,
  ms: number,
  message: string
): Promise<T> {
  return Promise.race([
    fn(),
    new Promise<never>((_, reject) =>
      setTimeout(() => reject(new Error(message)), ms)
    ),
  ]);
}

Path resolution after normalization is the non-obvious check. A schema that rejects .. in the raw string still needs a post-resolution check because some OS path functions normalize encoded sequences. Resolve the path first, then verify it is still inside the allowed root.

For network-capable tools, the allowedHosts check belongs at the HTTP client level, not the schema level. A model argument that passes schema validation can still carry an attacker-controlled URL. Apply the allowlist at the moment the request fires, not when arguments are parsed.

Tradeoffs: Validation Strategies

Dimension	Schema Validation Only	Schema + Pre-Execute Hook	Schema + Pre-Execute + Sandbox
Protection against malformed inputs	Yes	Yes	Yes
Protection against valid-but-wrong arguments	No	Yes	Yes
Protection against prompt injection	No	Partial	Yes
Implementation overhead	Low	Medium	High
Latency overhead	~0ms	1-10ms	5-50ms
Appropriate for	Read-only, low-stakes tools	Mutation tools with authorization	Filesystem, shell, network tools

Do not apply the same validation level to every tool. A read-only search tool that returns paginated results needs structural validation. A file deletion tool needs all three layers.

Retry Strategies Specific to Agent Loops

General-purpose retry libraries (exponential backoff, jitter) solve network-layer failures. Agent loops have a different class of failures that require different retry logic.

In an agent loop, a tool failure is not necessarily an infrastructure problem. It might be a logic error in the model’s reasoning: wrong tool chosen, argument generated from a false assumption, a tool called out of order. Retrying the same call with the same arguments will not fix it. The model needs the error fed back as context so it can revise its approach.

The distinction matters:

• Transient failure (network timeout, rate limit): retry the call transparently, increment a counter, do not change agent state
• Argument error (schema rejection, authorization denial): return the error as a tool result, let the model revise
• Loop detection (same tool called with identical arguments more than N times): halt the loop, escalate

interface AgentLoopConfig {
  maxTurns: number;
  maxTransientRetries: number;
  transientRetryDelayMs: number;
  loopDetectionWindow: number; // last N tool calls to check for repetition
}

interface ToolCallRecord {
  name: string;
  inputHash: string;
  turnIndex: number;
}

function hashInput(input: unknown): string {
  return require("crypto")
    .createHash("sha256")
    .update(JSON.stringify(input))
    .digest("hex")
    .slice(0, 16);
}

function detectLoop(
  history: ToolCallRecord[],
  windowSize: number
): boolean {
  if (history.length < windowSize) return false;
  const window = history.slice(-windowSize);
  const unique = new Set(window.map((r) => `${r.name}:${r.inputHash}`));
  // If fewer than half the calls in the window are unique, treat as a loop
  return unique.size < Math.ceil(windowSize / 2);
}

async function executeWithTransientRetry(
  fn: () => Promise<{ success: boolean; result: unknown; error?: string }>,
  config: AgentLoopConfig
): Promise<{ success: boolean; result: unknown; error?: string }> {
  let attempt = 0;
  while (attempt <= config.maxTransientRetries) {
    const result = await fn();
    if (result.success) return result;
    // Only retry on transient errors, not argument errors
    const isTransient =
      result.error?.includes("timeout") ||
      result.error?.includes("rate limit") ||
      result.error?.includes("503") ||
      result.error?.includes("529");
    if (!isTransient || attempt === config.maxTransientRetries) return result;
    attempt++;
    await new Promise((r) => setTimeout(r, config.transientRetryDelayMs * 2 ** attempt));
  }
  // Unreachable, but TypeScript requires it
  return { success: false, result: null, error: "Max retries exceeded" };
}

The loopDetectionWindow check is the one most implementations skip. A model stuck in a reasoning loop will call the same tool with the same arguments repeatedly. Without this check, your agent burns through tokens and potentially triggers expensive external calls until maxTurns cuts it off. With it, you detect the loop early and surface a useful error.

Observability for Tool Execution Chains

Tool calls in an agent loop form a chain: each call depends on prior results, and a failure at any point affects all downstream calls. Standard request-level metrics do not capture this. You need per-tool-call instrumentation that retains the chain structure.

The minimum useful data for each tool call:

• Tool name and input hash (not the full input: inputs can contain sensitive data)
• Execution duration in milliseconds
• Success or failure
• Error category if failed (schema, auth, transient, unknown)
• Turn index and call index within the turn
• Session ID and agent run ID for correlation

interface ToolCallSpan {
  agentRunId: string;
  sessionId: string;
  turnIndex: number;
  callIndex: number;
  toolName: string;
  inputHash: string;
  startedAt: number; // unix ms
  durationMs: number;
  success: boolean;
  errorCategory?: "schema" | "auth" | "rate_limit" | "timeout" | "business_rule" | "unknown";
  outputSizeBytes?: number;
}

function categorizeError(error: string | undefined): ToolCallSpan["errorCategory"] {
  if (!error) return undefined;
  if (error.includes("Schema validation")) return "schema";
  if (error.includes("Unauthorized") || error.includes("denied")) return "auth";
  if (error.includes("rate limit") || error.includes("limit:")) return "rate_limit";
  if (error.includes("timeout")) return "timeout";
  if (error.includes("Pre-execution check")) return "business_rule";
  return "unknown";
}

async function executeWithObservability(
  call: NormalizedToolCall,
  tool: RegisteredTool,
  ctx: ToolExecutionContext,
  agentRunId: string,
  callIndex: number,
  emit: (span: ToolCallSpan) => void
): Promise<{ success: boolean; result: unknown; error?: string }> {
  const start = Date.now();
  const result = await executeToolCall(call, tool, ctx);
  const durationMs = Date.now() - start;

  const outputStr = JSON.stringify(result.result ?? "");
  const span: ToolCallSpan = {
    agentRunId,
    sessionId: ctx.sessionId,
    turnIndex: ctx.turnIndex,
    callIndex,
    toolName: call.name,
    inputHash: hashInput(call.input),
    startedAt: start,
    durationMs,
    success: result.success,
    errorCategory: categorizeError(result.error),
    outputSizeBytes: Buffer.byteLength(outputStr, "utf8"),
  };
  emit(span);

  return result;
}

Do not log the full input to your metrics pipeline. Tool inputs frequently contain user data (emails, names, query strings) that should not end up in logs. Log the hash for correlation and deduplication, and log the full input only to an append-only audit store with appropriate access controls if you need it for debugging.

The metrics worth tracking at the aggregate level:

• Tool error rate by category: schema errors trending up means your prompts or schemas changed. Auth errors trending up means either your prompt injection controls are weakening or a legitimate use case is being blocked.
• Tool call latency P50/P95/P99 per tool: a single slow tool will dominate agent turn latency.
• Loop detection trigger rate: if this is non-zero in production, find the prompt or tool set that is causing it.
• Calls per session per tool: outliers indicate a model stuck in a retry pattern that your loop detection missed.
• Output size P95 per tool: unexpectedly large outputs inflate context window usage and cost.

Production Considerations

Tool catalog versioning. The schema your tool presents to the model is part of your API contract. Changing a field name or removing an enum value is a breaking change: agents trained or prompted with the old schema will generate arguments that fail validation. Version your tool schemas the same way you version external APIs. Deploy schema changes with a backward-compatible transition period rather than cutting over immediately.

Context poisoning through tool results. Tool results go back into the conversation context. A malicious external system (a document the agent fetched, a database row it retrieved) can include text that attempts to override the system prompt or hijack tool calls. This is the “prompt injection via tool result” attack surface. Treat tool results as untrusted data: strip or escape markup, limit the result size that goes back into context, and consider a summarization step for large external results before they re-enter the context window.

Idempotency for mutation tools. Agent loops retry. If a turn fails mid-execution (LLM API timeout, your server crash), the loop may replay tool calls that already executed. Mutation tools need idempotency keys. Generate the key before the loop starts, derive it from the session and turn index, and pass it through to your downstream systems. The tool handler should accept the key and make the operation idempotent on replay.

Cost attribution. In an agent with a large tool catalog, it is easy to lose track of what is actually driving token cost. Tool descriptions, tool results, and the accumulated conversation context each contribute. Instrument the context size (in tokens) at the start of each turn, the total tokens in tool results returned that turn, and the LLM API cost per turn. A tool that returns verbose results is often the single largest cost driver.

Timeout budgets across the full turn. Each tool call has its own timeout. But the agent turn as a whole also needs a budget. If you have five parallel tool calls each with a 10-second timeout, a single slow call can hold the turn open for 10 seconds even if the others finish in 200ms. Set an outer turn timeout that is shorter than the sum of individual tool timeouts, and cancel remaining calls when it fires.

The Layer as Infrastructure

The shift from glue code to infrastructure is a decision about ownership. Glue code has no owner: it lives in the agent file, grows through copy-paste, and breaks silently. An infrastructure layer has an interface, versioning, observability, and failure modes that are understood and handled.

The concrete form of that layer: a tool registry with typed definitions, a validated execution pipeline that every tool call passes through, a sandbox gate for dangerous tools, loop detection in the agent loop, and structured spans for every call. None of this is complex. Together, it means the next agent you build starts with safe defaults rather than requiring you to re-solve the same problems from scratch.

The problems you will not encounter if you build this once: a schema change silently breaking a deployed agent, an authorization bypass that creates a CRUD vulnerability, a model stuck in a loop burning $40 of API calls in a single session, a tool result containing user PII ending up in your APM dashboard.

Build the layer. Add tools to it. The agents come after.