Llm Context Window Engineering: Diff-Scoped Context Windows That Catch Real Bugs

AI PR Reviewers: Diff-Scoped Context Windows That Catch Real Bugs
16 views

This guide covers LLM context window engineering with production-focused examples and trade-offs.

Full-file context makes AI PR bots verbose and blind: they nibble import order while missing an authz bypass in a callee two hops away. The unfair advantage is diff-scoped windows: changed symbols, call-graph neighbors, and contract tests only — then ask for nullability, authorization, and schema risks. Humans still own merge; the bot owns high-signal nits that would otherwise escape.
⚡ TL;DR: Build context from git diff + symbol extraction + 1–2 hop callers/callees + related *.test.ts / OpenAPI slices. Cap tokens hard. Prompt for ranked findings with file:line and severity. Drop style nits CI already covers. Wire secrets/IAM carefully (AI code review bots). Align with Cursor monorepo rules.

Build the context pack from the diff — LLM context window engineering

typescript
// ci/ai-review/context.ts
import { execSync } from "child_process";

export function changedFiles(base: string): string[] {
  return execSync(`git diff --name-only ${base}...HEAD`, { encoding: "utf8" })
    .trim()
    .split("\n")
    .filter((f) => /\.(ts|tsx|js|py)$/.test(f));
}

export function unifiedDiff(base: string, file: string): string {
  // ✅ Prefer unified diff hunks over whole files
  return execSync(`git diff -U5 ${base}...HEAD -- ${file}`, { encoding: "utf8" });
}
typescript
// ❌ Dumping entire files "so the model has context"
const context = files.map((f) => readFileSync(f, "utf8")).join("\n\n");

Expand to call-graph neighbors (bounded)

typescript
export type SymbolHit = { file: string; name: string; kind: "fn" | "class" };

export function neighborContext(hits: SymbolHit[], maxHops = 1, tokenBudget = 6000): string {
  const parts: string[] = [];
  let used = 0;
  for (const h of hits) {
    const callees = lookupCallees(h).slice(0, 5); // ✅ hard cap
    const callers = lookupCallers(h).slice(0, 5);
    for (const n of [...callees, ...callers]) {
      const snippet = readSymbolSource(n);
      if (used + snippet.length > tokenBudget) return parts.join("\n\n");
      parts.push(`// neighbor ${n.file}::${n.name}\n${snippet}`);
      used += snippet.length;
    }
  }
  return parts.join("\n\n");
}
Include contract tests that mention the same symbols — that is where schema and authz regressions show up.

Prompt for bugs CI cannot see

text
You are a senior PR reviewer. Given DIFF hunks, NEIGHBORS, and TESTS:
- Find nullability, authz, tenancy, and schema/contract breaks
- Cite file:line and severity (blocker/major/nit)
- Ignore formatting, import order, and naming already enforced by eslint
- If unsure, say uncertain — do not invent APIs
Return JSON: { findings: [{ severity, file, line, title, why }] }
typescript
type Finding = {
  severity: "blocker" | "major" | "nit";
  file: string;
  line: number;
  title: string;
  why: string;
};

export function filterFindings(raw: Finding[]): Finding[] {
  // ✅ Drop nits; keep blockers/majors for human attention
  return raw.filter((f) => f.severity !== "nit" && f.line > 0);
}

Post comments without spamming

One summary comment + inline notes on blockers. Dedipe against previous bot runs on the same SHA.
bash
# ✅ Idempotent review marker
gh api repos/$REPO/commits/$SHA/comments -f body="$(cat review.md)"
Security posture for tokens and least privilege: AI Code Review Bots: IAM, Secrets, and Least-Privilege Pipelines. Sandbox any tool execution: LLM Coding Agents on AWS.

What “good” looks like in metrics

Metric Illust. target
Precision of blocker findings ≥ 0.6 (humans agree)
Comments per PR ≤ 8
Style nits filed ~0 (eslint owns them)
Time-to-first-bot-review < 3 min on affected packs

Closing checklist

✅ Dos
– ✅ Context = diff hunks + bounded neighbors + contract tests
– ✅ Hard token budget; JSON findings with file:line
– ✅ Filter to blocker/major; suppress eslint-covered nits
– ✅ Least-privilege GitHub App tokens
– ✅ Track precision with human feedback ❌ Don’ts
– ❌ Don’t send entire unchanged files
– ❌ Don’t let the bot bikeshed formatting
– ❌ Don’t auto-approve merges from the model
– ❌ Don’t expand call graph unbounded
– ❌ Don’t paste secrets from CI into the model prompt

Related reading

Last updated on September 11, 2026

Related: Day 20: Project: PR Review Bot With Diff-Scoped Context · Unit Testing Best Practices: Write Tests That Actually Catch Bugs Before Production · Day 1: Tokens, Context Windows, and Why Models Forget Mid-Task

Related: AI Test Generation: Property Tests That Catch Race Conditions · Secret-Aware Context Filters: Stop AI Editors From Shipping Keys · Python typing Module: Generics and Protocols That Catch Real Bugs at Compile Time

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

Comments

No comments yet. Why don’t you start the discussion?

Leave a comment

No account needed. Name and email are optional.