Building AI Copilots for Domain-Specific Applications: Context Retrieval, Action Execution, and User Trust in Vertical AI

A generic AI assistant is fine for drafting emails. It is not fine for reviewing a healthcare prior authorization, flagging a contract clause that violates jurisdictional regulations, or recommending a draw schedule on a commercial construction project. The difference is not model quality. It is domain context, action scope, and the tolerance for error.

Horizontal AI tools treat every question the same way: embed the query, retrieve some documents, and generate a response. Vertical copilots treat the domain as a first-class citizen. They know the difference between a lien waiver and a mechanics lien. They know that a “patient discharge” in one context and a “policy discharge” in another use the same word for entirely different objects. They are built around the constraints of the domain, not bolted onto a generic chatbot frame.

This article covers the four engineering layers that make a vertical copilot production-ready: context retrieval, action execution, user trust calibration, and the compliance logging layer that regulated industries require.

Why Horizontal AI Tools Fail in Vertical Domains

The failure mode is always the same: the model does not have the right context and the user cannot tell.

A legal associate asks a horizontal AI tool about a clause in a merger agreement. The tool retrieves some generic contract law content, generates a confident response, and the associate ships the advice. The nuance that mattered (jurisdiction-specific carve-outs, a precedent from a specific court) was not in the retrieval pool. There was no signal to the user that the confidence was misplaced.

Vertical copilots fail differently from horizontal ones, and that difference is what you are building against:

• Structured data is the actual context source. In healthcare, the relevant context is not a document chunk. It is a patient record, a formulary, a prior auth rule set, and a payer contract, all structured. RAG over free text misses 60% of the actual decision-relevant context.
• Actions have consequences. A horizontal chatbot cannot send a message, update a record, or trigger a workflow. A vertical copilot often can, and wrong actions in regulated industries create liability, not just embarrassment.
• Confidence calibration is a compliance requirement. In some domains, an AI system that presents uncertain output as certain violates regulatory guidance. The system needs to know what it does not know.

Context Retrieval Architecture

Standard RAG is a good starting point. It is not sufficient for vertical domains.

Combining RAG with Domain Knowledge Graphs

The architecture that works in practice combines three retrieval paths and fuses them at query time.

interface RetrievalContext {
  chunks: DocumentChunk[];         // vector search results
  graphFacts: KnowledgeGraphNode[]; // structured entity relationships
  structuredRecords: Record<string, unknown>[]; // database rows (EHR, CRM, ERP)
  retrievalScores: {
    chunkRelevance: number;
    graphCoverage: number;
    structuredFreshness: number;
  };
}

interface DocumentChunk {
  id: string;
  content: string;
  source: string;
  embeddingScore: number;
  metadata: {
    domain: string;
    entityRefs: string[];   // IDs of entities mentioned — links to graph
    validFrom: string;
    jurisdiction?: string;
  };
}

interface KnowledgeGraphNode {
  entityId: string;
  entityType: string;    // "Regulation", "Clause", "Patient", "Project"
  properties: Record<string, unknown>;
  relationships: Array<{
    type: string;
    targetId: string;
    weight: number;
  }>;
}

async function retrieveDomainContext(
  query: string,
  domainScope: DomainScope,
  sessionContext: SessionContext,
): Promise<RetrievalContext> {
  // Run three retrievals in parallel
  const [chunks, graphFacts, structuredRecords] = await Promise.all([
    vectorSearch(query, { filter: domainScope, topK: 12 }),
    graphSearch(extractEntities(query, domainScope), { depth: 2 }),
    structuredLookup(sessionContext.entityIds, domainScope.schema),
  ]);

  // Entity refs in chunks link to graph — expand them
  const expandedChunks = await expandGraphRefs(chunks, graphFacts);

  return {
    chunks: expandedChunks,
    graphFacts,
    structuredRecords,
    retrievalScores: scoreRetrieval(chunks, graphFacts, structuredRecords, query),
  };
}

The key design decision here is the entityRefs field on each chunk. When you index documents, you run a domain NER pass and tag every chunk with the IDs of the domain entities it mentions. At retrieval time, you expand those refs against the knowledge graph to pull in current structured state, not just the text that was true when the document was indexed.

In a legal context, a clause chunk might reference reg:SEC-10b-5 and jurisdiction:NY. At query time you expand those refs to get current enforcement status, recent case citations, and any superseding guidance. The chunk alone would be stale. The graph-expanded chunk is current.

Prompt Construction for Vertical Context

The context you assembled needs to be laid out for the model in a way that preserves the structure. Dumping all three retrieval paths into a single text blob loses the signal about which facts are structured (authoritative) vs. retrieved (probabilistic).

function buildVerticalPrompt(
  query: string,
  context: RetrievalContext,
  domain: DomainConfig,
): string {
  const structuredSection = context.structuredRecords.length > 0
    ? `<structured_data authority="high">\n${formatStructured(context.structuredRecords)}\n</structured_data>`
    : '';

  const graphSection = context.graphFacts.length > 0
    ? `<knowledge_graph authority="high">\n${formatGraph(context.graphFacts)}\n</knowledge_graph>`
    : '';

  const chunksSection = context.chunks
    .slice(0, 8)  // limit to top-8 after reranking
    .map((c, i) => `<document id="${i}" source="${c.source}" relevance="${c.embeddingScore.toFixed(2)}">\n${c.content}\n</document>`)
    .join('\n');

  return `${domain.systemPrompt}

DOMAIN: ${domain.name}
JURISDICTION: ${domain.jurisdiction ?? 'not specified'}

${structuredSection}
${graphSection}

<retrieved_documents>
${chunksSection}
</retrieved_documents>

<query>${query}</query>

Respond only based on the above context. If the answer requires information not present in the context, say so explicitly and indicate what additional information would be needed. Do not infer facts not supported by the context.`;
}

The explicit authority tags are not decorative. They condition the model to weight structured records over retrieved chunks when they conflict, which they will in production.

Action Execution

A copilot that only answers questions is an expensive search engine. The value in vertical AI comes from the copilot taking actions: drafting a document, submitting a prior auth, flagging a contract for review, or updating a project schedule.

Tool-Use Patterns

The pattern that holds up in production is: tools are narrow, typed, and idempotent where possible. Do not give the model a “do database thing” tool. Give it fetchPatientMedications, submitPriorAuthRequest, and flagClauseForReview.

// Tool registry — each tool has explicit schema and risk level
interface CopilotTool {
  name: string;
  description: string;
  riskLevel: 'read' | 'write-reversible' | 'write-irreversible' | 'external';
  requiresApproval: boolean;
  inputSchema: z.ZodSchema;
  outputSchema: z.ZodSchema;
  execute: (input: unknown, context: ExecutionContext) => Promise<ToolResult>;
}

interface ToolResult {
  success: boolean;
  data: unknown;
  auditEntry: AuditEntry;  // every tool call generates an audit entry
}

interface AuditEntry {
  toolName: string;
  input: unknown;
  output: unknown;
  executedAt: string;
  executedBy: string;   // user ID — the human who approved, or system for auto
  sessionId: string;
  traceId: string;
  approved: boolean;
  approvalLatencyMs?: number;
}

// Example: a write tool that requires human approval
const submitPriorAuthRequest: CopilotTool = {
  name: 'submitPriorAuthRequest',
  description: 'Submit a prior authorization request to the payer on behalf of the provider',
  riskLevel: 'write-irreversible',
  requiresApproval: true,
  inputSchema: z.object({
    patientId: z.string(),
    procedureCode: z.string(),
    diagnosisCodes: z.array(z.string()),
    clinicalNotes: z.string(),
    payerId: z.string(),
  }),
  outputSchema: z.object({
    authorizationNumber: z.string().optional(),
    status: z.enum(['submitted', 'pending', 'approved', 'denied']),
    referenceId: z.string(),
  }),
  execute: async (input, context) => {
    const validated = submitPriorAuthRequest.inputSchema.parse(input);
    const result = await payerApi.submitAuth(validated, context.credentials);
    return {
      success: true,
      data: result,
      auditEntry: {
        toolName: 'submitPriorAuthRequest',
        input: validated,
        output: result,
        executedAt: new Date().toISOString(),
        executedBy: context.approvedBy,
        sessionId: context.sessionId,
        traceId: context.traceId,
        approved: true,
        approvalLatencyMs: context.approvalLatencyMs,
      },
    };
  },
};

Approval Workflows

The approval layer is where most implementations underinvest. The pattern is a short-lived approval token: the copilot proposes an action with a rendered preview, the user approves or rejects, and the approval token is attached to the execution context. No approval token, no write execution.

interface ApprovalRequest {
  requestId: string;
  sessionId: string;
  toolName: string;
  proposedInput: unknown;
  humanReadableSummary: string;  // copilot explains what it is about to do
  riskLevel: CopilotTool['riskLevel'];
  expiresAt: string;  // short TTL — 5 minutes is reasonable
  status: 'pending' | 'approved' | 'rejected' | 'expired';
}

async function executeWithApproval(
  tool: CopilotTool,
  proposedInput: unknown,
  context: SessionContext,
): Promise<ToolResult> {
  if (!tool.requiresApproval) {
    return tool.execute(proposedInput, buildExecutionContext(context, null));
  }

  const approvalRequest: ApprovalRequest = {
    requestId: crypto.randomUUID(),
    sessionId: context.sessionId,
    toolName: tool.name,
    proposedInput,
    humanReadableSummary: await generateActionSummary(tool, proposedInput),
    riskLevel: tool.riskLevel,
    expiresAt: new Date(Date.now() + 5 * 60 * 1000).toISOString(),
    status: 'pending',
  };

  await approvalStore.create(approvalRequest);
  await notifyUserForApproval(approvalRequest, context.userId);

  // Poll or subscribe to approval status
  const approval = await waitForApproval(approvalRequest.requestId, {
    timeoutMs: 5 * 60 * 1000,
    pollIntervalMs: 1000,
  });

  if (approval.status !== 'approved') {
    throw new ApprovalRejectedError(approvalRequest.requestId, approval.status);
  }

  const execContext = buildExecutionContext(context, {
    approvedBy: context.userId,
    approvalLatencyMs: Date.now() - new Date(approvalRequest.expiresAt).getTime() + 5 * 60 * 1000,
  });

  return tool.execute(proposedInput, execContext);
}

One non-obvious decision: generate the humanReadableSummary using the same LLM that proposed the action, but with a separate prompt that asks it to explain what it is doing in plain language and flag anything the user should verify. This surfaces the model’s own uncertainty in the approval flow rather than hiding it.

User Trust Calibration

The hardest part of vertical AI is calibrating user trust correctly. Too much trust and users follow the copilot off a cliff. Too little trust and no one uses it.

Confidence Scoring

Confidence scoring that is honest requires measuring three things independently and then combining them.

interface ConfidenceScores {
  retrievalCoverage: number;   // 0-1: did retrieval find enough relevant context?
  answerGroundedness: number;  // 0-1: is the answer supported by retrieved context?
  entityResolution: number;    // 0-1: did all entities resolve cleanly in the graph?
  combined: number;
  flags: ConfidenceFlag[];
}

type ConfidenceFlag =
  | 'low-retrieval-coverage'
  | 'unresolved-entities'
  | 'conflicting-sources'
  | 'jurisdiction-mismatch'
  | 'out-of-date-precedent'
  | 'missing-structured-record';

async function scoreConfidence(
  query: string,
  context: RetrievalContext,
  response: string,
): Promise<ConfidenceScores> {
  const retrievalCoverage = context.retrievalScores.chunkRelevance;
  const entityResolution = context.retrievalScores.graphCoverage;

  // Check answer groundedness with a separate LLM call
  // Ask: "Is this answer fully supported by the provided context?
  //        List any claims not grounded in the context."
  const groundednessResult = await checkGroundedness(query, context, response);

  const flags: ConfidenceFlag[] = [];
  if (retrievalCoverage < 0.6) flags.push('low-retrieval-coverage');
  if (entityResolution < 0.7) flags.push('unresolved-entities');
  if (groundednessResult.ungroundedClaims.length > 0) flags.push('conflicting-sources');

  const combined =
    retrievalCoverage * 0.35 +
    groundednessResult.score * 0.45 +
    entityResolution * 0.2;

  return { retrievalCoverage, answerGroundedness: groundednessResult.score, entityResolution, combined, flags };
}

The weights (0.35, 0.45, 0.20) are a starting point. In a healthcare context you might weight entity resolution more heavily because unresolved patient or medication entities are a safety issue. In a construction context, retrieval coverage matters more because the relevant documents are often project-specific and sparse.

Progressive Autonomy

Autonomy should increase with demonstrated accuracy, and decrease when flags accumulate. The simplest version tracks per-user per-domain confidence history and adjusts the default approval threshold.

Confidence Combined	Flags Present	Default Action
> 0.85	None	Auto-execute (read tools) / Present with low friction (write tools)
0.65-0.85	None	Present with explanation, one-click approve
0.65-0.85	Any flag	Present with flag explanation, explicit confirm required
< 0.65	Any	Offer response as draft only, require manual review before any action
< 0.50	Any	Surface uncertainty explicitly, do not offer action path

This table belongs in your product configuration, not hardcoded. Different vertical deployments have different risk tolerances, and you will tune these thresholds based on observed error rates in production.

The Compliance Layer

In legal, healthcare, and financial services, the logging requirement is not optional. Every decision the copilot makes, every action it takes or proposes, and every piece of context it used must be reconstructable from the audit log.

interface CopilotDecisionRecord {
  id: string;
  sessionId: string;
  tenantId: string;
  userId: string;
  timestamp: string;
  query: string;
  retrievalContext: {
    chunkIds: string[];
    graphNodeIds: string[];
    structuredRecordRefs: string[];
    retrievalScores: RetrievalContext['retrievalScores'];
  };
  modelUsed: string;
  promptHash: string;        // hash of the full prompt — lets you reproduce exactly
  response: string;
  confidenceScores: ConfidenceScores;
  actionsProposed: Array<{
    toolName: string;
    proposedInput: unknown;
    approvalStatus: ApprovalRequest['status'];
    approvalRequestId?: string;
    executedAt?: string;
    result?: unknown;
  }>;
  regulatoryFlags: string[];  // any flags raised by domain-specific compliance rules
}

async function persistDecisionRecord(record: CopilotDecisionRecord): Promise<void> {
  // Write to append-only store — records must not be mutable after write
  await auditDb.insert('copilot_decisions', record);

  // Separately index for compliance queries
  await complianceIndex.index({
    id: record.id,
    tenantId: record.tenantId,
    timestamp: record.timestamp,
    flags: record.regulatoryFlags,
    hasActions: record.actionsProposed.length > 0,
  });
}

Two non-negotiables in the compliance layer:

1. Records are append-only. Do not update them. If you need to add an outcome (approval status, action result), write a linked record with a relatedDecisionId field.
2. The promptHash field lets you reconstruct the exact input the model received. Store the full prompt text separately, keyed by hash, so you can reproduce any decision for audit review without storing duplicate prompt text per record.

For HIPAA-regulated healthcare deployments, the retrievalContext.structuredRecordRefs field must be specific enough that an auditor can verify exactly which patient records the model accessed during a decision. Vague references will not satisfy an OCR audit.

Architecture Tradeoffs

Concern	Simple approach	Robust approach	When to upgrade
Context retrieval	Flat vector search over document chunks	Hybrid: vector + knowledge graph + structured lookup	When domain has relational entities (patients, contracts, projects) that documents reference
Action execution	Free-form LLM-generated tool calls	Typed tool registry with schema validation and approval layer	From day one in any regulated domain
Confidence scoring	Single embedding similarity score	Multi-dimensional: coverage + groundedness + entity resolution	When users are making consequential decisions on AI output
Audit logging	Application logs with request/response	Append-only structured record with full context reconstruction	Any domain with regulatory or liability exposure
Autonomy level	Fixed threshold for all users	Per-user per-domain history-weighted threshold	After you have enough history to calibrate per user

Production Considerations

Knowledge graph maintenance is an ongoing cost. The graph is only as good as the last update. In legaltech, regulations change. In healthcare, formularies update monthly. Build ingestion pipelines that detect changes to source documents and trigger targeted graph updates, not full rebuilds.

Tool schema version pinning matters. When you update a tool’s input schema, existing pending approvals become invalid. Version your tools explicitly (submitPriorAuthRequest@v2) and handle schema migration in the approval store.

Prompt hash collisions are a compliance risk. If two different prompts hash to the same value (unlikely but non-zero), an auditor following the hash will reconstruct the wrong prompt. Use SHA-256 and store the full prompt text in a content-addressed store keyed by hash. Collision resistance at SHA-256 is sufficient for audit purposes.

Latency compounds in the retrieval layer. Three parallel retrievals (vector, graph, structured) will each have tail latencies. Build a timeout per retrieval path with a fallback: if the graph lookup times out, proceed with vector results and set graphCoverage to 0 in the confidence scores. The confidence score then flags the gap, and the user sees lower confidence rather than a timeout error.

Test your confidence scores against labeled examples. Every vertical domain has subject-matter experts. Build a small labeled eval set (100-200 queries with known correct answers and known confidence levels) and run it against your scoring pipeline weekly. Confidence score drift is a real production issue, especially after model updates.

The Vertical AI Opportunity

Vertical AI platforms capture more than 40% of AI investment in 2026, and the reason is straightforward: generic AI tools generate generic output, and domain buyers are paying for domain expertise embedded in the system, not just an LLM wrapper.

The architecture described here is not cheap to build. A knowledge graph, typed tool registry, multi-dimensional confidence scoring, and append-only compliance logging add real engineering overhead compared to a RAG-over-PDFs chatbot. That overhead is the moat. The teams who build it correctly for a specific vertical (legal ops, clinical workflows, construction project management) end up with a system that a generic AI tool cannot replicate by prompt engineering alone, because the context and action layers are deeply entangled with domain-specific data sources and workflows.

Get the context retrieval right first. That is where the most leverage is, and it is where horizontal tools fail most visibly. The rest follows.