Building a RAG Pipeline for Code Search: Repository Indexing, Semantic Retrieval, and Context Assembly for AI Development Tools

Most teams building internal AI development tools reach for the same stack they used for document Q&A: chunk the files, embed the chunks, store them in a vector database, retrieve by similarity. It works well enough in a demo. Then someone asks the AI assistant about a bug involving three files, two interface definitions, and a helper that was renamed two weeks ago, and the retrieved context is completely wrong.

Code is not a document. The structural properties that make code meaningful are precisely the things that naive text chunking destroys.

This article covers how to build a RAG pipeline that treats code as code: chunking at semantically meaningful boundaries, selecting embedding models that understand syntax, indexing metadata that enables targeted filtering, assembling context that gives an LLM enough structure to reason about the code correctly, and keeping the index current as the repository evolves.

Why Code Search Is Different

Document search fails on code for three concrete reasons.

Syntax makes chunk boundaries semantic. Splitting a document at 500 characters is arbitrary but roughly harmless. Splitting a TypeScript file at 500 characters almost certainly cuts through a function, a class, or an interface. The resulting chunks are syntactically broken and semantically disconnected from their neighbors.

Cross-file dependencies are load-bearing. A function call in one file means nothing without the type signature from a different file. A React component only makes sense alongside the hook that manages its state. When you retrieve the call site without the definition, or the component without the hook, the LLM has to guess. It guesses wrong.

Recency matters more than in document search. A design document from six months ago might still be authoritative. A function implementation from six months ago was probably refactored. Stale code context is worse than no context because the LLM treats it as current.

These three problems require specific architectural responses: AST-aware chunking, metadata that captures cross-file relationships, and incremental indexing tied to git history.

Chunking Strategies

The goal is chunks that are syntactically complete and semantically self-contained.

Function-Level Chunking

The smallest useful unit for most code retrieval is the function or method. It has a clear start and end, a defined signature, and a bounded purpose. Function-level chunks are also small enough to fit many into a context window.

import Parser from "tree-sitter";
import TypeScript from "tree-sitter-typescript";

interface CodeChunk {
  id: string;
  content: string;
  filePath: string;
  language: string;
  chunkType: "function" | "class" | "interface" | "module";
  symbolName: string;
  startLine: number;
  endLine: number;
  exports: boolean;
  dependencies: string[]; // imported symbols used in this chunk
}

async function extractFunctionChunks(
  filePath: string,
  source: string
): Promise<CodeChunk[]> {
  const parser = new Parser();
  parser.setLanguage(TypeScript.typescript);
  const tree = parser.parse(source);
  const chunks: CodeChunk[] = [];

  function walk(node: Parser.SyntaxNode): void {
    const functionTypes = [
      "function_declaration",
      "method_definition",
      "arrow_function",
      "function_expression",
    ];

    if (functionTypes.includes(node.type)) {
      const name = node.childForFieldName("name")?.text ?? "anonymous";
      const chunkSource = source.slice(node.startIndex, node.endIndex);

      chunks.push({
        id: `${filePath}:${name}:${node.startPosition.row}`,
        content: chunkSource,
        filePath,
        language: "typescript",
        chunkType: "function",
        symbolName: name,
        startLine: node.startPosition.row,
        endLine: node.endPosition.row,
        exports: isExported(node),
        dependencies: extractImportedSymbols(node, source),
      });
    }

    node.children.forEach(walk);
  }

  walk(tree.rootNode);
  return chunks;
}

The dependencies field is important. When you retrieve a function chunk, you know which imported symbols it uses. That lets you optionally expand the context to include those definitions before sending to the LLM.

Class-Level and Interface-Level Chunking

For classes and interfaces, function-level chunking is too granular. A class with fifteen methods should be retrievable as a unit so the LLM can understand the object’s contract.

The practical approach is a two-level strategy: index each class as a whole chunk, and also index each method within it as a child chunk. The class chunk carries the full context; the method chunks support more targeted retrieval. Store the parent class ID as metadata on each method chunk so you can expand upward when needed.

File-Level Chunking as a Fallback

Some files do not decompose cleanly: configuration files, barrel exports, type-only files. Index these at the file level. They are typically small enough to fit in a single chunk, and their content is only meaningful as a whole.

What Not to Do

Overlapping sliding windows work reasonably well for prose but poorly for code. The overlap creates redundant chunks without semantic coherence. A function appears in multiple chunks, each syntactically broken at different points. The embedding is noisy and retrieval quality drops. Stick to AST-aware boundaries.

Embedding Model Selection

Three categories of models are worth considering, each with different tradeoffs.

Code-Specific Models

CodeBERT (Microsoft) was trained on GitHub code in six languages. It produces 768-dimensional embeddings with strong understanding of identifier names, code structure, and comment relationships. It underperforms on very long functions and on newer language features not in its training data.

StarCoder embeddings (from the StarCoder2 family) have broader language coverage and better handling of modern TypeScript, including decorators, generic constraints, and complex type expressions. If your codebase is TypeScript-heavy, this is worth evaluating.

Both are open-source and can be self-hosted, which matters when you are indexing a proprietary codebase and do not want to send source code to external APIs.

General-Purpose Models

OpenAI text-embedding-3-small and 3-large produce high-quality embeddings for code despite not being trained exclusively on it. The 3-large model with 3072 dimensions shows strong performance on code similarity tasks and benefits from OpenAI’s broad training data. The tradeoff: your source code leaves your infrastructure, and cost scales with repository size.

Comparison

Model	Dimensions	Language coverage	Self-hostable	Cost at 1M tokens
CodeBERT	768	6 languages	Yes	Infrastructure only
StarCoder2-embed	768	80+ languages	Yes	Infrastructure only
text-embedding-3-small	1536	Broad	No	~$0.02
text-embedding-3-large	3072	Broad	No	~$0.13

For most teams: use a self-hosted StarCoder or CodeBERT model if your codebase is sensitive, use text-embedding-3-small if convenience matters more than data residency.

The practical recommendation is to evaluate on your own repository. Embed 200 functions, write 50 queries that represent real developer questions, measure recall@5 for each model. The results often differ from published benchmarks because benchmark distributions rarely match your actual codebase.

Vector Store Indexing with Metadata

The embedding captures semantic similarity. Metadata enables filtering that pure vector search cannot do.

interface CodeChunkMetadata {
  filePath: string;
  language: string;
  chunkType: "function" | "class" | "interface" | "module";
  symbolName: string;
  startLine: number;
  endLine: number;
  exports: boolean;
  packageName?: string;
  gitCommitHash: string;
  lastModifiedAt: number; // unix timestamp
  author?: string;
  tags: string[]; // e.g., ["auth", "database", "api"]
}

async function indexChunk(
  chunk: CodeChunk,
  embedding: number[],
  commitHash: string,
  vectorStore: VectorStoreClient
): Promise<void> {
  const metadata: CodeChunkMetadata = {
    filePath: chunk.filePath,
    language: chunk.language,
    chunkType: chunk.chunkType,
    symbolName: chunk.symbolName,
    startLine: chunk.startLine,
    endLine: chunk.endLine,
    exports: chunk.exports,
    gitCommitHash: commitHash,
    lastModifiedAt: Date.now(),
    tags: inferTags(chunk.filePath, chunk.content),
  };

  await vectorStore.upsert({
    id: chunk.id,
    values: embedding,
    metadata,
  });
}

function inferTags(filePath: string, content: string): string[] {
  const tags: string[] = [];
  if (filePath.includes("/auth/") || content.includes("authentication")) tags.push("auth");
  if (filePath.includes("/db/") || content.includes("prisma") || content.includes("knex")) tags.push("database");
  if (filePath.includes("/api/") || filePath.includes("/routes/")) tags.push("api");
  if (filePath.includes(".test.") || filePath.includes(".spec.")) tags.push("test");
  return tags;
}

With this metadata, retrieval queries can include filters:

async function searchCode(
  query: string,
  options: {
    language?: string;
    filePath?: string;
    chunkType?: string;
    tags?: string[];
    sinceCommit?: string; // filter to chunks modified after a specific commit
  },
  vectorStore: VectorStoreClient,
  embedModel: EmbeddingModel
): Promise<CodeChunk[]> {
  const queryEmbedding = await embedModel.embed(query);

  const filter: Record<string, unknown> = {};
  if (options.language) filter.language = { $eq: options.language };
  if (options.chunkType) filter.chunkType = { $eq: options.chunkType };
  if (options.tags?.length) filter.tags = { $in: options.tags };
  if (options.filePath) filter.filePath = { $eq: options.filePath };

  const results = await vectorStore.query({
    vector: queryEmbedding,
    topK: 20,
    filter,
    includeMetadata: true,
  });

  return results.matches.map(deserializeChunk);
}

The topK: 20 is intentional. You retrieve more than you will use, then re-rank.

Retrieval Pipeline with Re-Ranking

Vector similarity is a coarse filter, not a final ranking. The top result by cosine similarity is often not the most useful chunk for the actual question.

Re-ranking works by taking the top-k candidates from vector search and scoring each one against the query using a cross-encoder model. Cross-encoders process the query and the document together, which is too expensive to run over the entire index but tractable over 20 candidates.

import Anthropic from "@anthropic-ai/sdk";

interface RankedChunk {
  chunk: CodeChunk;
  vectorScore: number;
  rerankScore: number;
  finalScore: number;
}

async function rerankChunks(
  query: string,
  candidates: CodeChunk[],
  llm: Anthropic
): Promise<RankedChunk[]> {
  const scoringPromises = candidates.map(async (chunk, index) => {
    const prompt = `Rate the relevance of this code snippet to the query on a scale from 0 to 1.

Query: ${query}

Code snippet (${chunk.symbolName} in ${chunk.filePath}):
\`\`\`${chunk.language}
${chunk.content.slice(0, 800)}
\`\`\`

Respond with a JSON object: {"score": <number between 0 and 1>, "reason": "<brief reason>"}`;

    const response = await llm.messages.create({
      model: "claude-3-haiku-20240307",
      max_tokens: 100,
      messages: [{ role: "user", content: prompt }],
    });

    const parsed = JSON.parse(
      (response.content[0] as { type: "text"; text: string }).text
    );

    return {
      chunk,
      vectorScore: 1 - index / candidates.length, // normalized position score
      rerankScore: parsed.score as number,
      finalScore: 0, // computed below
    };
  });

  const ranked = await Promise.all(scoringPromises);

  return ranked
    .map((r) => ({
      ...r,
      finalScore: 0.3 * r.vectorScore + 0.7 * r.rerankScore,
    }))
    .sort((a, b) => b.finalScore - a.finalScore)
    .slice(0, 5); // take the top 5 after re-ranking
}

Using a small, fast LLM (Haiku-class or equivalent) for re-ranking keeps latency reasonable. The key insight is that the re-ranker only needs to score 20 short snippets, not generate a full answer. Total re-ranking latency should be under 2 seconds with parallel requests.

Context Window Assembly

The goal of context assembly is to give the LLM enough code to answer correctly without exceeding the context window or drowning the signal in noise.

interface AssembledContext {
  primaryChunks: CodeChunk[];
  expandedDefinitions: CodeChunk[];
  totalTokens: number;
  truncated: boolean;
}

async function assembleContext(
  topChunks: RankedChunk[],
  query: string,
  tokenBudget: number,
  indexStore: CodeChunkStore
): Promise<AssembledContext> {
  const primaryChunks = topChunks.map((r) => r.chunk);
  const expandedDefinitions: CodeChunk[] = [];
  let usedTokens = 0;

  // First pass: add primary chunks
  const fittedPrimary: CodeChunk[] = [];
  for (const chunk of primaryChunks) {
    const chunkTokens = estimateTokens(chunk.content);
    if (usedTokens + chunkTokens <= tokenBudget * 0.7) {
      fittedPrimary.push(chunk);
      usedTokens += chunkTokens;
    }
  }

  // Second pass: expand dependencies if budget remains
  const remainingBudget = tokenBudget - usedTokens;
  const seenIds = new Set(fittedPrimary.map((c) => c.id));

  for (const chunk of fittedPrimary) {
    for (const dep of chunk.dependencies) {
      if (seenIds.has(dep)) continue;

      const depChunk = await indexStore.lookupBySymbol(dep);
      if (!depChunk) continue;

      const depTokens = estimateTokens(depChunk.content);
      if (usedTokens + depTokens <= tokenBudget) {
        expandedDefinitions.push(depChunk);
        seenIds.add(dep);
        usedTokens += depTokens;
      }
    }
  }

  return {
    primaryChunks: fittedPrimary,
    expandedDefinitions,
    totalTokens: usedTokens,
    truncated: fittedPrimary.length < primaryChunks.length,
  };
}

function formatContextForLLM(ctx: AssembledContext, query: string): string {
  const sections: string[] = [];

  if (ctx.expandedDefinitions.length > 0) {
    sections.push("### Referenced Definitions\n");
    for (const chunk of ctx.expandedDefinitions) {
      sections.push(
        `// ${chunk.filePath} (${chunk.symbolName})\n\`\`\`${chunk.language}\n${chunk.content}\n\`\`\``
      );
    }
  }

  sections.push("### Relevant Code\n");
  for (const chunk of ctx.primaryChunks) {
    sections.push(
      `// ${chunk.filePath}:${chunk.startLine} (${chunk.symbolName})\n\`\`\`${chunk.language}\n${chunk.content}\n\`\`\``
    );
  }

  return sections.join("\n\n");
}

The 70% budget split for primary chunks is deliberate. If you allocate the full budget to primary chunks, there is no room for dependency expansion, which often determines whether the answer is correct.

One practical note: include file paths and line numbers in the formatted context. The LLM uses these to reason about where code lives and whether two chunks are related. Without them, the model cannot distinguish a function from src/auth/session.ts from one with the same name in src/api/session.ts.

Production Considerations

Incremental Indexing on Git Push

A full re-index on every push is expensive and unnecessary. Git provides a precise diff of what changed.

import { execSync } from "child_process";

interface GitDiff {
  added: string[];
  modified: string[];
  deleted: string[];
}

function getChangedFiles(fromCommit: string, toCommit: string): GitDiff {
  const diffOutput = execSync(
    `git diff --name-status ${fromCommit} ${toCommit}`
  ).toString();

  const added: string[] = [];
  const modified: string[] = [];
  const deleted: string[] = [];

  for (const line of diffOutput.trim().split("\n")) {
    const [status, filePath] = line.split("\t");
    if (!filePath) continue;

    if (status === "A") added.push(filePath);
    else if (status === "M") modified.push(filePath);
    else if (status === "D") deleted.push(filePath);
  }

  return { added, modified, deleted };
}

async function incrementalIndex(
  diff: GitDiff,
  newCommitHash: string,
  indexer: CodeIndexer,
  vectorStore: VectorStoreClient
): Promise<void> {
  // Delete chunks from removed files
  for (const filePath of diff.deleted) {
    await vectorStore.deleteWhere({ filePath: { $eq: filePath } });
  }

  // Re-index modified and added files
  const toIndex = [...diff.added, ...diff.modified];
  for (const filePath of toIndex) {
    // Remove old chunks for this file before re-indexing
    if (diff.modified.includes(filePath)) {
      await vectorStore.deleteWhere({ filePath: { $eq: filePath } });
    }

    const source = await readFile(filePath);
    const chunks = await indexer.chunk(filePath, source);
    const embeddings = await indexer.embed(chunks);

    for (let i = 0; i < chunks.length; i++) {
      await indexChunk(chunks[i], embeddings[i], newCommitHash, vectorStore);
    }
  }
}

Trigger this as a webhook on push. Your CI system receives the push event, extracts the before/after commit hashes, and runs incrementalIndex. For most repositories, this completes in under 30 seconds.

Stale Index Detection

Even with incremental indexing, indexes go stale in specific ways. Deleted symbol references accumulate over time as functions are renamed or removed. The dependencies field on chunks points to symbols that may no longer exist.

Run a nightly validation job that checks a random sample of indexed chunks:

async function validateIndexSample(
  sampleSize: number,
  vectorStore: VectorStoreClient,
  repo: GitRepository
): Promise<{ stale: number; total: number }> {
  const sample = await vectorStore.randomSample(sampleSize);
  let staleCount = 0;

  for (const chunk of sample) {
    const currentContent = await repo.readFileAtHead(
      chunk.metadata.filePath
    ).catch(() => null);

    if (!currentContent) {
      // File was deleted but chunk was not removed
      staleCount++;
      await vectorStore.delete(chunk.id);
      continue;
    }

    const symbolStillExists = currentContent.includes(
      chunk.metadata.symbolName
    );
    if (!symbolStillExists) {
      staleCount++;
      await vectorStore.delete(chunk.id);
    }
  }

  return { stale: staleCount, total: sampleSize };
}

If the stale rate exceeds 5%, trigger a full re-index. This is a self-healing mechanism for cases where the incremental indexer missed a batch of changes or the schema changed.

Chunking Strategy Tradeoffs

Strategy	Query types it handles well	Failure modes	Index size
Function-level AST	Specific function questions, bug investigation	Misses module-level patterns	Medium
Class-level AST	OOP design questions, interface contracts	Loses method-level detail	Medium
File-level	Configuration, barrel exports, small utilities	Too large for focused retrieval	Large
Sliding window (text)	Works with any language	Syntactically broken, noisy embeddings	Large
Hybrid (function + class + file)	Broad coverage	Deduplication required, more complex indexer	Largest

Most production setups benefit from the hybrid approach, but start with function-level only. It covers 80% of developer queries and is straightforward to implement correctly.

What This Gets You

A code RAG pipeline built this way changes the failure mode of AI development tools from “the answer is confidently wrong” to “the answer is sometimes incomplete.” The difference matters in practice. A developer can work with an answer that says “I found the session creation logic in auth/session.ts, but the downstream token validation happens somewhere I did not retrieve.” They cannot work with an answer that confidently describes the wrong function.

The architectural decisions that drive this are not the embedding model choice or the vector store vendor. They are the chunking boundaries, the metadata schema, and the dependency expansion during context assembly. Those three decisions determine whether the LLM sees enough of the right code to reason correctly. Everything else is plumbing.