AI PR Reviewers: Diff-Scoped Context Windows That Catch Real Bugs

AI PR Reviewers: Diff-Scoped Context Windows That Catch Real Bugs

Full-file context makes AI PR bots verbose and blind: they nibble import order while missing an authz bypass in a callee two hops away. The unfair advantage is diff-scoped windows: changed symbols, call-graph neighbors, and contract tests only — then ask for nullability, authorization, and schema risks. Humans still own merge; the bot owns high-signal nits that would otherwise escape.

⚡ TL;DR: Build context from git diff + symbol extraction + 1–2 hop callers/callees + related *.test.ts / OpenAPI slices. Cap tokens hard. Prompt for ranked findings with file:line and severity. Drop style nits CI already covers. Wire secrets/IAM carefully (AI code review bots). Align with Cursor monorepo rules.

Build the context pack from the diff

// ci/ai-review/context.ts
import { execSync } from "child_process";

export function changedFiles(base: string): string[] {
  return execSync(`git diff --name-only ${base}...HEAD`, { encoding: "utf8" })
    .trim()
    .split("\n")
    .filter((f) => /\.(ts|tsx|js|py)$/.test(f));
}

export function unifiedDiff(base: string, file: string): string {
  // ✅ Prefer unified diff hunks over whole files
  return execSync(`git diff -U5 ${base}...HEAD -- ${file}`, { encoding: "utf8" });
}
// ❌ Dumping entire files "so the model has context"
const context = files.map((f) => readFileSync(f, "utf8")).join("\n\n");

Expand to call-graph neighbors (bounded)

export type SymbolHit = { file: string; name: string; kind: "fn" | "class" };

export function neighborContext(hits: SymbolHit[], maxHops = 1, tokenBudget = 6000): string {
  const parts: string[] = [];
  let used = 0;
  for (const h of hits) {
    const callees = lookupCallees(h).slice(0, 5); // ✅ hard cap
    const callers = lookupCallers(h).slice(0, 5);
    for (const n of [...callees, ...callers]) {
      const snippet = readSymbolSource(n);
      if (used + snippet.length > tokenBudget) return parts.join("\n\n");
      parts.push(`// neighbor ${n.file}::${n.name}\n${snippet}`);
      used += snippet.length;
    }
  }
  return parts.join("\n\n");
}

Include contract tests that mention the same symbols — that is where schema and authz regressions show up.

Prompt for bugs CI cannot see

You are a senior PR reviewer. Given DIFF hunks, NEIGHBORS, and TESTS:
- Find nullability, authz, tenancy, and schema/contract breaks
- Cite file:line and severity (blocker/major/nit)
- Ignore formatting, import order, and naming already enforced by eslint
- If unsure, say uncertain — do not invent APIs
Return JSON: { findings: [{ severity, file, line, title, why }] }
type Finding = {
  severity: "blocker" | "major" | "nit";
  file: string;
  line: number;
  title: string;
  why: string;
};

export function filterFindings(raw: Finding[]): Finding[] {
  // ✅ Drop nits; keep blockers/majors for human attention
  return raw.filter((f) => f.severity !== "nit" && f.line > 0);
}

Post comments without spamming

One summary comment + inline notes on blockers. Dedipe against previous bot runs on the same SHA.

# ✅ Idempotent review marker
gh api repos/$REPO/commits/$SHA/comments -f body="$(cat review.md)"

Security posture for tokens and least privilege: AI Code Review Bots: IAM, Secrets, and Least-Privilege Pipelines. Sandbox any tool execution: LLM Coding Agents on AWS.

What “good” looks like in metrics

Metric Illust. target
Precision of blocker findings ≥ 0.6 (humans agree)
Comments per PR ≤ 8
Style nits filed ~0 (eslint owns them)
Time-to-first-bot-review < 3 min on affected packs

Closing checklist

✅ Dos
– ✅ Context = diff hunks + bounded neighbors + contract tests
– ✅ Hard token budget; JSON findings with file:line
– ✅ Filter to blocker/major; suppress eslint-covered nits
– ✅ Least-privilege GitHub App tokens
– ✅ Track precision with human feedback

❌ Don’ts
– ❌ Don’t send entire unchanged files
– ❌ Don’t let the bot bikeshed formatting
– ❌ Don’t auto-approve merges from the model
– ❌ Don’t expand call graph unbounded
– ❌ Don’t paste secrets from CI into the model prompt

Related reading

Last updated on September 11, 2026


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply