This guide covers semantic diff review with production-focused examples and trade-offs.
Most AI PR comments are noise: “consider renaming this variable,” “add a blank line,” “imports aren’t sorted.” Seniors mute the bot. Semantic diff review flips the objective — ignore formatting-only churn and spend the context window on AST-level and behavioral drift: contract breaks, authz regressions, and performance cliffs.⚡ TL;DR: Preprocess the PR into a semantic diff (AST-aware, ignore whitespace/import reorder). Feed the reviewer changed symbols + callers + contract tests only. Prompt for severity-tagged findings with reproducible evidence. Drop nits below a threshold. Illustrative win: 70% fewer comments, higher accept rate on high-severity findings.
Build a semantic diff, not a unified diff dump
# Prefer structural diffs when available
git diff --ignore-all-space origin/main...HEAD > /tmp/raw.diff
# Illustrative: emit changed symbols via scip / lsif / ts-morph
node tools/semantic-diff.js --base origin/main --out /tmp/semantic.json
// semantic-diff.js excerpt — focus on behavioral surface
import { Project } from "ts-morph";
export function changedExports(baseFiles: string[], headFiles: Map<string, string>) {
const findings: { path: string; name: string; kind: string }[] = [];
const project = new Project({ skipAddingFilesFromTsConfig: true });
for (const [path, text] of headFiles) {
const sf = project.createSourceFile(path, text, { overwrite: true });
for (const [name, decls] of sf.getExportedDeclarations()) {
findings.push({ path, name, kind: decls[0].getKindName() });
}
}
return findings;
}
❌ Pass: prettier-only files, import resort, generated lockfile blobs.
Context packing for the reviewer model
Follow the diff-scoped pattern: changed symbols + one-hop callers + existing tests.{
"pr": 1842,
"behavioral_diff": [
{
"symbol": "InvoiceService.create",
"signature_before": "(input: CreateInvoice) => Promise<Invoice>",
"signature_after": "(input: CreateInvoice, opts?: {skipTax?: boolean}) => Promise<Invoice>",
"callers": ["CheckoutController.complete"],
"tests": ["invoice.service.test.ts"]
}
],
"contracts": ["openapi.yaml#/paths/~1invoices/post"],
"forbidden_nits": ["style", "naming-bike-shed", "import-order"]
}
You are a senior code reviewer. Only report:
1) contract/API breaks 2) authz/authn regressions 3) data-loss or idempotency bugs
4) obvious p99 performance cliffs 5) secret leakage.
Ignore formatting, naming taste, and optional refactors.
Severity: blocker|major|minor. Require file:line evidence.
If no behavioral risk, reply NO_FINDINGS.
Auth and contract checks that beat vibes
# fail the review job if OpenAPI response schema drifted without changelog
import json, sys
before, after = json.load(open(sys.argv[1])), json.load(open(sys.argv[2]))
# illustrative: compare /invoices post 200 schema hash
assert before["paths"]["/invoices"]["post"]["responses"]["200"] == \
after["paths"]["/invoices"]["post"]["responses"]["200"], "schema drift — require dual-run note"
Suppress nits ruthlessly
# reviewer-config.yml
max_comments: 8
drop_severities: [nit]
block_on: [blocker, major]
ignore_paths:
- "**/*.generated.*"
- "**/pnpm-lock.yaml"
prompt_version: semantic-review@2026-09-11
Wire into CI without blocking every PR forever
Run semantic review as a non-blocking check that posts comments, and only fail the build onblocker findings the team has calibrated for two weeks.
# .github/workflows/semantic-review.yml — illustrative
name: semantic-review
on: pull_request
jobs:
review:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- run: node tools/semantic-diff.js --base origin/${{ github.base_ref }} --out semantic.json
- run: node tools/run-reviewer.js --in semantic.json --prompt-version semantic-review@2026-09-11
env:
BEDROCK_MODEL: ${{ vars.REVIEW_MODEL }}
BUDGET_PR_TOKENS: "200000"
Closing checklist
✅ Dos – ✅ Precompute semantic/AST diffs before prompting – ✅ Include callers and contract artifacts in context – ✅ Severity-tag findings; auto-drop nits – ✅ Version the reviewer prompt; A/B on real PRs – ✅ Measure human accept rate, not comment volume ❌ Don’ts – ❌ Don’t paste the entire unified diff into the model – ❌ Don’t allow style bikesheds to page authors – ❌ Don’t grant the bot monorepo admin tokens – ❌ Don’t treat “LGTM from AI” as a required check alone – ❌ Don’t review generated files without an allowlist exceptionRelated reading
- AI Code Review Bots: IAM, Secrets, and Least-Privilege Pipelines
- Cursor Rules for TypeScript Monorepos: Make AI Edits Stick
- Agentic Git Workflows: Atomic Commits From Noisy LLM Diffs (companion)
- LLM Coding Agents on AWS: Safe Tool Sandboxes with Lambda
Related: Spec-First AI Development: OpenAPI Remains the Only Source of Truth · AI Code Review Automation: Build a GitHub Bot That Reviews PRs Like a Senior Engineer · Day 93: SLOs for AI Features
Related: Prompt Engineering for Developers: The Patterns That 10x Your AI Coding Output · Vibe Coding: How Senior Developers Actually Use AI to Write Code Faster · Day 80: Project: Platform AI Gateway
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
