Building LLM-Powered Code Review Pipelines: Static Analysis, AI Review, and Developer Workflow Integration

The gap between “we have a code review bot” and “our code review bot is actually useful” is almost entirely architectural. The demo is easy: take a diff, send it to an LLM, post the response as a PR comment. The production system is where it gets hard. You need to decide where the pipeline runs (git hooks, CI, or both), how to handle diffs that exceed context windows, whether to block merges automatically, how to measure whether the reviews are improving over time, and how to keep the cost under control as PR volume grows.

This article walks through each of those decisions with concrete TypeScript code and tradeoff tables. It assumes you already have a working LLM integration and focuses on the pipeline: the orchestration layer that makes AI code review a reliable part of your developer workflow rather than an experiment that gets ignored after the first week.

---

Where the Pipeline Lives

The first architectural decision is where the pipeline runs. You have three options, and they are not mutually exclusive.

Git hooks (pre-push or pre-receive): Run analysis on the engineer’s machine or on the server before the push completes. Fast feedback, but limited compute and timeout constraints.

CI pipeline (GitHub Actions, GitLab CI, etc.): Runs in a controlled environment, can take longer, has access to the full repo, and integrates with PR workflows natively.

Dedicated webhook service: A long-running service that receives PR events and processes them asynchronously, decoupled from CI run time.

For most teams, the right answer is: lightweight fast checks in a pre-push hook, and full LLM review in CI. The hook catches obvious problems locally without paying LLM inference costs on every push. CI runs the deeper analysis once and posts inline comments.

Git Hook Integration

A pre-push hook can run local static analysis and a fast, cheap LLM pass over only the files touched in the current push. The key constraint is that hooks must be fast (under 30 seconds) or engineers will disable them.

// .git/hooks/pre-push (sourced via a Node script)
// scripts/pre-push-check.ts

import { execSync } from "child_process";
import { readFileSync } from "fs";

interface ChangedFile {
  path: string;
  additions: number;
}

function getLocalChangedFiles(baseSha: string, headSha: string): ChangedFile[] {
  const output = execSync(
    `git diff --numstat ${baseSha}..${headSha}`
  ).toString();

  return output
    .split("\n")
    .filter(Boolean)
    .map((line) => {
      const [additions, , path] = line.split("\t");
      return { path, additions: parseInt(additions, 10) };
    })
    .filter((f) => /\.(ts|tsx|js|jsx|py|go)$/.test(f.path));
}

async function runPrePushCheck(baseSha: string, headSha: string): Promise<void> {
  const files = getLocalChangedFiles(baseSha, headSha);

  // Skip if this is a large change - let CI handle it
  if (files.length > 20) {
    console.log(`[review] ${files.length} files changed. Skipping pre-push check; CI will review.`);
    return;
  }

  const totalAdditions = files.reduce((sum, f) => sum + f.additions, 0);
  if (totalAdditions > 500) {
    console.log(`[review] Large diff (${totalAdditions} additions). Skipping pre-push; CI will review.`);
    return;
  }

  // Lightweight local lint only - no LLM in the hook
  try {
    execSync(`npx tsc --noEmit 2>&1 | head -20`, { stdio: "inherit" });
  } catch {
    console.error("[review] TypeScript errors found. Fix before pushing.");
    process.exit(1);
  }

  console.log("[review] Pre-push checks passed. Full AI review will run in CI.");
}

// Entry point called from the hook script
const [, , baseSha, headSha] = process.argv;
runPrePushCheck(baseSha, headSha).catch((err) => {
  console.error("[review] Pre-push check failed:", err.message);
  // Non-zero exit blocks the push
  process.exit(1);
});

Keep the hook focused on type-checking and fast linting. Save LLM calls for CI where you control the environment and can retry failures.

---

CI Pipeline Integration

In CI, the review runs as a step after the test suite passes. This ordering matters: there is no point reviewing code that does not pass tests.

A minimal GitHub Actions workflow:

# .github/workflows/ai-review.yml
name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  review:
    runs-on: ubuntu-latest
    # Only run if tests passed (reference your test job)
    needs: [test]
    permissions:
      pull-requests: write
      contents: read
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - uses: actions/setup-node@v4
        with:
          node-version: "20"

      - name: Install dependencies
        run: npm ci

      - name: Run AI review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          PR_NUMBER: ${{ github.event.pull_request.number }}
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: npx ts-node scripts/ci-review.ts

The CI review script has access to the full checkout, can run git diff directly, and uses the built-in GITHUB_TOKEN to post comments:

// scripts/ci-review.ts

import { execSync } from "child_process";

const PR_NUMBER = parseInt(process.env.PR_NUMBER!, 10);
const BASE_SHA = process.env.BASE_SHA!;
const HEAD_SHA = process.env.HEAD_SHA!;
const REPO = process.env.GITHUB_REPOSITORY!; // "owner/repo"
const TOKEN = process.env.GITHUB_TOKEN!;

async function fetchDiff(): Promise<string> {
  // Use git directly in CI - avoids GitHub API rate limits
  return execSync(`git diff ${BASE_SHA}..${HEAD_SHA}`).toString();
}

async function getPRTitle(): Promise<string> {
  const resp = await fetch(
    `https://api.github.com/repos/${REPO}/pulls/${PR_NUMBER}`,
    { headers: { Authorization: `Bearer ${TOKEN}` } }
  );
  const pr = await resp.json() as { title: string };
  return pr.title;
}

---

Handling Large Diffs: Chunking Strategies

Context windows cap out. Even models with large windows perform worse as context fills up - the model’s attention degrades on early content when prompts exceed ~30k tokens. You need a chunking strategy for large diffs.

The naive approach is to split by file. Better is to split by semantic unit: file plus its immediate dependency context. The best is to split by review concern: group related files together (e.g., a controller and its validation schema).

interface DiffChunk {
  id: string;
  files: FileDiff[];
  estimatedTokens: number;
  reviewFocus: string; // human-readable description of what this chunk covers
}

function chunkDiffForReview(
  files: FileDiff[],
  maxTokensPerChunk = 12_000
): DiffChunk[] {
  const chunks: DiffChunk[] = [];
  let current: FileDiff[] = [];
  let currentTokens = 0;
  let chunkIndex = 0;

  // Group test files with their implementation counterparts
  const groupedFiles = groupRelatedFiles(files);

  for (const group of groupedFiles) {
    const groupTokens = group.reduce(
      (sum, f) => sum + estimateFileTokens(f),
      0
    );

    if (currentTokens + groupTokens > maxTokensPerChunk && current.length > 0) {
      chunks.push({
        id: `chunk-${chunkIndex++}`,
        files: current,
        estimatedTokens: currentTokens,
        reviewFocus: describeChunk(current),
      });
      current = [];
      currentTokens = 0;
    }

    current.push(...group);
    currentTokens += groupTokens;
  }

  if (current.length > 0) {
    chunks.push({
      id: `chunk-${chunkIndex}`,
      files: current,
      estimatedTokens: currentTokens,
      reviewFocus: describeChunk(current),
    });
  }

  return chunks;
}

function groupRelatedFiles(files: FileDiff[]): FileDiff[][] {
  const groups: Map<string, FileDiff[]> = new Map();

  for (const file of files) {
    // Strip test suffixes and extensions to find the base module name
    const base = file.path
      .replace(/\.(test|spec)\.(ts|tsx|js|jsx)$/, "")
      .replace(/\.(ts|tsx|js|jsx)$/, "");

    if (!groups.has(base)) groups.set(base, []);
    groups.get(base)!.push(file);
  }

  return Array.from(groups.values());
}

function estimateFileTokens(file: FileDiff): number {
  const content = file.hunks.flatMap((h) => h.lines).join("\n");
  // ~3.5 characters per token is a reasonable estimate for code
  return Math.ceil(content.length / 3.5);
}

function describeChunk(files: FileDiff[]): string {
  const paths = files.map((f) => f.path);
  if (paths.length === 1) return `Review of ${paths[0]}`;
  const dirs = [...new Set(paths.map((p) => p.split("/").slice(0, -1).join("/")))];
  return dirs.length === 1
    ? `Review of ${dirs[0]} (${paths.length} files)`
    : `Review of ${paths.length} files across ${dirs.length} directories`;
}

For each chunk, run a separate LLM call and aggregate the results. When you aggregate, deduplicate by file+line before posting. The same off-by-one bug in a shared utility might appear in multiple chunks.

---

Confidence Scoring and Auto-Approve Logic

Not every PR needs a human to wait on AI review comments. Define a confidence score to decide when to auto-approve, when to flag for human review, and when to block.

interface ReviewScore {
  confidence: number;         // 0-1: how confident the model is in its findings
  issueCount: number;
  criticalCount: number;
  majorCount: number;
  staticErrorCount: number;
  diffSize: "small" | "medium" | "large";
  decision: "approve" | "comment" | "request-changes";
}

function scoreReview(
  review: ReviewResponse,
  staticIssues: StaticIssue[],
  fileDiffs: FileDiff[]
): ReviewScore {
  const criticalCount = review.comments.filter((c) => c.severity === "critical").length;
  const majorCount = review.comments.filter((c) => c.severity === "major").length;
  const staticErrorCount = staticIssues.filter((i) => i.severity === "error").length;

  const totalAdditions = fileDiffs.reduce((sum, f) => sum + f.additions, 0);
  const diffSize =
    totalAdditions < 100 ? "small" : totalAdditions < 500 ? "medium" : "large";

  // Confidence degrades with diff size (model attention degrades on long contexts)
  // and with static errors (static errors suggest lower-quality code overall)
  let confidence = 1.0;
  if (diffSize === "medium") confidence -= 0.1;
  if (diffSize === "large") confidence -= 0.25;
  if (staticErrorCount > 0) confidence -= 0.15;
  // Multiple chunks were reviewed separately - aggregation may miss cross-file issues
  if (fileDiffs.length > 10) confidence -= 0.1;
  confidence = Math.max(0, Math.min(1, confidence));

  let decision: ReviewScore["decision"];
  if (criticalCount > 0 || staticErrorCount > 0) {
    decision = "request-changes";
  } else if (majorCount > 0 || confidence < 0.7) {
    decision = "comment"; // Post comments but don't block
  } else {
    decision = "approve"; // High confidence, no significant issues
  }

  return {
    confidence,
    issueCount: review.comments.length,
    criticalCount,
    majorCount,
    staticErrorCount,
    diffSize,
    decision,
  };
}

Auto-approve only when confidence is high and there are no significant issues:

async function submitFinalDecision(
  token: string,
  repo: string,
  prNumber: number,
  headSha: string,
  score: ReviewScore,
  review: ReviewResponse
): Promise<void> {
  const eventMap = {
    "approve": "APPROVE",
    "comment": "COMMENT",
    "request-changes": "REQUEST_CHANGES",
  } as const;

  const body = buildReviewBody(score, review);

  await fetch(
    `https://api.github.com/repos/${repo}/pulls/${prNumber}/reviews`,
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${token}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        commit_id: headSha,
        body,
        event: eventMap[score.decision],
        comments: review.comments
          .filter((c) => c.severity !== "nit")
          .map((c) => ({
            path: c.file,
            line: c.line,
            side: "RIGHT",
            body: `**[${c.severity.toUpperCase()}]** ${c.message}${c.suggestion ? `\n\n\`\`\`\n${c.suggestion}\n\`\`\`` : ""}`,
          })),
      }),
    }
  );
}

function buildReviewBody(score: ReviewScore, review: ReviewResponse): string {
  const statusLine =
    score.decision === "approve"
      ? `Automated review passed (confidence: ${Math.round(score.confidence * 100)}%)`
      : score.decision === "request-changes"
      ? `Issues require attention before merging.`
      : `Review complete. Comments posted for consideration.`;

  return [
    statusLine,
    "",
    review.summary,
    "",
    `Diff size: ${score.diffSize} | Issues: ${score.issueCount} (${score.criticalCount} critical, ${score.majorCount} major) | Static errors: ${score.staticErrorCount}`,
    "",
    "_Automated review. Verify suggestions before applying._",
  ].join("\n");
}

Set conservative auto-approve thresholds to start. It is better to over-comment than to auto-approve a PR with a race condition the model missed because the diff was too large. Expand auto-approve coverage as you validate the confidence scoring against real outcomes.

---

GitLab API Integration

If your team uses GitLab, the pipeline logic is identical but the API surface differs. The key differences are: GitLab uses “merge requests” not “pull requests”, uses a different authentication model (project access tokens or CI_JOB_TOKEN), and posts inline comments differently.

interface GitLabMREvent {
  object_kind: "merge_request";
  object_attributes: {
    iid: number;             // MR number within the project
    state: "opened" | "updated" | "merged" | "closed";
    last_commit: { id: string };
    target_branch: string;
    source_branch: string;
    action: "open" | "update" | "reopen";
  };
  project: {
    id: number;
    path_with_namespace: string;
  };
}

async function fetchGitLabDiff(
  projectId: number,
  mrIid: number,
  token: string
): Promise<string> {
  const resp = await fetch(
    `https://gitlab.com/api/v4/projects/${projectId}/merge_requests/${mrIid}/diffs`,
    { headers: { "PRIVATE-TOKEN": token } }
  );
  const diffs = await resp.json() as Array<{
    diff: string;
    new_path: string;
    old_path: string;
    new_file: boolean;
    deleted_file: boolean;
  }>;

  // Reconstruct unified diff format for the parser
  return diffs
    .map(
      (d) =>
        `diff --git a/${d.old_path} b/${d.new_path}\n` +
        (d.new_file ? "new file mode 100644\n" : "") +
        (d.deleted_file ? "deleted file mode 100644\n" : "") +
        d.diff
    )
    .join("\n");
}

async function postGitLabMRComment(
  projectId: number,
  mrIid: number,
  token: string,
  comment: ReviewComment,
  headSha: string
): Promise<void> {
  // GitLab uses "discussions" for inline comments
  await fetch(
    `https://gitlab.com/api/v4/projects/${projectId}/merge_requests/${mrIid}/discussions`,
    {
      method: "POST",
      headers: {
        "PRIVATE-TOKEN": token,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        body: `**[${comment.severity.toUpperCase()}]** ${comment.message}`,
        position: {
          position_type: "text",
          base_sha: headSha,         // GitLab requires three SHAs for positioning
          start_sha: headSha,
          head_sha: headSha,
          new_path: comment.file,
          new_line: comment.line,
        },
      }),
    }
  );
}

async function approveGitLabMR(
  projectId: number,
  mrIid: number,
  token: string
): Promise<void> {
  await fetch(
    `https://gitlab.com/api/v4/projects/${projectId}/merge_requests/${mrIid}/approve`,
    {
      method: "POST",
      headers: { "PRIVATE-TOKEN": token },
    }
  );
}

GitLab’s CI/CD environment exposes CI_JOB_TOKEN which has read access to the repository but not MR write access. Use a dedicated project access token with api scope for posting reviews. Store it as a masked CI variable.

---

Cost Management

LLM review costs compound fast at scale. At 50 PRs per day, even $0.10 per review is $1,500/month.

interface CostBudget {
  maxInputTokensPerPR: number;   // Caps context sent per review
  maxOutputTokensPerPR: number;  // Caps response length
  dailyBudgetUSD: number;        // Kills reviews if exceeded
  modelTier: "fast" | "standard" | "premium";
}

const MODEL_COSTS_PER_1K = {
  // Approximate costs as of early 2026 - update these as pricing changes
  fast: { input: 0.0001, output: 0.0004 },       // Small/fast models
  standard: { input: 0.0003, output: 0.0012 },   // Mid-tier models
  premium: { input: 0.003, output: 0.015 },       // Frontier models
} as const;

function estimateCost(
  inputTokens: number,
  outputTokens: number,
  tier: CostBudget["modelTier"]
): number {
  const rates = MODEL_COSTS_PER_1K[tier];
  return (inputTokens / 1000) * rates.input + (outputTokens / 1000) * rates.output;
}

function selectModelForDiff(
  diffSize: "small" | "medium" | "large",
  hasSecurityChanges: boolean
): { model: string; tier: CostBudget["modelTier"] } {
  // Use cheaper models for small, low-risk diffs
  if (diffSize === "small" && !hasSecurityChanges) {
    return { model: "gpt-4o-mini", tier: "fast" };
  }
  if (diffSize === "large" || hasSecurityChanges) {
    return { model: "gpt-4o", tier: "premium" };
  }
  return { model: "gpt-4o-mini", tier: "standard" };
}

function detectsSecurityChanges(files: FileDiff[]): boolean {
  const securityPatterns = [
    /auth/i, /permission/i, /token/i, /secret/i,
    /password/i, /crypt/i, /sanitize/i, /escape/i,
    /sql/i, /query/i, /exec/i,
  ];
  return files.some((f) =>
    securityPatterns.some((p) => p.test(f.path) || p.test(f.hunks.flatMap((h) => h.lines).join("\n")))
  );
}

Route security-sensitive diffs to your best model. Route small formatting or documentation changes to a cheap, fast model. This tiered routing cuts costs 40-60% without reducing review quality on the diffs that matter.

---

Tradeoffs

Decision	Option A	Option B	Recommendation
Pipeline location	Git hooks only	CI only	Both: hooks for type errors locally, CI for LLM review
Large diff handling	Skip large PRs	Chunk and review all	Chunk up to a max, skip remainder with explanation
Auto-approve threshold	Never auto-approve	Auto-approve on high confidence	Start conservative, expand based on false-negative rate
Model selection	Single model for all	Tiered by diff risk	Tiered: saves 40-60% cost with minimal quality loss
GitLab vs GitHub	Pick one	Abstract the API layer	Abstract: the review logic is identical, only posting differs
Review freshness	Review all pushes	Review only the final commit	Review final commit only; rebases flood the review with noise

---

Measuring Review Quality

A pipeline that runs but does not improve is a cost center. You need evaluation metrics to know whether the LLM reviews are worth running.

Track these metrics per review:

interface ReviewMetrics {
  prId: string;
  timestamp: Date;
  promptVersion: string;
  model: string;
  inputTokens: number;
  outputTokens: number;
  costUSD: number;
  commentsGenerated: number;
  commentsSuppressed: number;
  commentsPosted: number;
  decision: "approve" | "comment" | "request-changes";
  confidence: number;
  // Populated after human review is complete
  humanOverride?: "accepted-auto" | "overrode-approve" | "overrode-block";
  issuesActedOn?: number;    // How many AI comments did the author address?
  issuesDismissed?: number;  // How many did they dismiss?
}

The most important metric is humanOverride. If overrode-approve is high (human blocked a PR the bot approved), your auto-approve threshold is too aggressive. If overrode-block is high (human approved a PR the bot blocked), your confidence model is miscalibrated.

Track issuesActedOn vs issuesDismissed by loading PR review resolution data after merge:

async function collectPostMergeMetrics(
  repo: string,
  prNumber: number,
  token: string
): Promise<{ actedOn: number; dismissed: number }> {
  // Fetch review threads to see which bot comments were resolved
  const resp = await fetch(
    `https://api.github.com/repos/${repo}/pulls/${prNumber}/comments`,
    { headers: { Authorization: `Bearer ${token}` } }
  );
  const comments = await resp.json() as Array<{
    user: { login: string };
    body: string;
    in_reply_to_id?: number;
  }>;

  const botComments = comments.filter((c) =>
    c.user.login.endsWith("[bot]") && c.body.includes("Automated review")
  );

  // Replies to bot comments indicate the author engaged with them
  const replied = new Set(
    comments
      .filter((c) => c.in_reply_to_id && !c.user.login.endsWith("[bot]"))
      .map((c) => c.in_reply_to_id)
  );

  const actedOn = botComments.filter((c) => replied.has(/* comment id */ 0)).length;
  return { actedOn, dismissed: botComments.length - actedOn };
}

Plot these metrics over time, sliced by prompt version. A prompt update should show an improvement in issuesActedOn / commentsPosted. If the ratio does not change after a prompt update, you changed style but not substance.

---

Production Considerations

Idempotency at scale. In CI, the same push can trigger multiple workflow runs if workflows are re-run manually. Store (prId, headSha, promptVersion) as a composite key in a review log. Return the cached result if it already exists rather than re-running.

Handling line number drift. When a reviewer pushes a fixup commit, the line numbers in the original review no longer map to the current diff. GitHub silently ignores comments with invalid positions. GitLab returns a 400. Validate that each comment’s line number falls within the actual hunk range before posting. For chunked reviews, re-validate against the complete diff, not just the chunk.

Review fatigue management. Set a per-author weekly cap on automated comments. If the bot has posted 50 comments to the same engineer this week, switch to summary-only mode: one comment listing the issues without inline annotations. This is a human-factor concern, not an engineering one, but it determines whether the pipeline survives adoption.

Prompt versioning. Every review should log which prompt version produced it. When you update a prompt, shadow-run it on 20 historical PRs and compare output before rolling out. A prompt that produces more comments is not necessarily better. Compare against the issuesActedOn rate of the current version.

Timeout handling. LLM calls fail. GitHub Actions jobs time out. Design the pipeline so that a review failure does not block the PR. Post a fallback comment (“Automated review failed. Manual review required.”) and exit with code 0 so the CI check does not block the merge.

---

Tradeoffs at a Glance

Dimension	Conservative	Aggressive	Sweet spot
Auto-approve rate	0% (humans always review)	80% (approve unless critical)	30-50% for low-risk PRs
Comment volume	Post everything	Post only critical	Post major+ with a nit summary
Chunking strategy	One chunk per file	Max context per chunk	Group related files, 12k tokens/chunk
Model tier	Premium for all diffs	Fast model for all diffs	Tiered by security and size signal
Post-merge metrics	None	Full resolution tracking	Track override rate; weekly audit

---

A code review pipeline earns trust by being reliable and selective. The engineers who use it daily will remember the three comments that caught real bugs. They will not remember the forty nit comments they dismissed. Start with narrow scope: security-sensitive files, high-churn modules, files with no test coverage. Measure what happens, adjust thresholds, and expand from there. The pipeline that covers 30% of your PRs with 80% precision beats the one that covers 100% with 40% precision.