Bedrock Batch Inference: Overnight Refactor Diffs Across Large Repos

Mechanical refactors — rename a deprecated SDK client, migrate logger APIs, rewrite import paths — do not need interactive tokens at 2 p.m. They need cheap overnight capacity, deterministic prompts, and a merge strategy that will not brick main. Amazon Bedrock Batch Inference is built for deferred work; this guide shows how to turn it into stacked, reviewable PRs instead of a 4,000-file monster commit.

⚡ TL;DR: Chunk the repo into compile-safe shards, emit JSONL batch jobs with frozen prompts + model IDs, validate every proposed diff with tsc/tests offline, then open stacked PRs by package. Never auto-merge mass AI refactors. Use Batch Inference for cost; keep interactive Converse for hot paths. Illustrative win: cut per-file token cost ~50% vs on-demand while humans review ~200-line PRs instead of a weekend-long diff.

When batch beats interactive agents

Use overnight batch when:

• The transform is mechanical and schema-constrained (AST or regex-safe intents)
• Failure is cheap if a shard is rejected
• You can gate on compile + unit tests without human pairing per file

Stay interactive when the change needs design judgment, multi-file behavioral redesign, or security-sensitive IAM edits.

Prompt caching helps interactive prefixes (Bedrock Prompt Caching and Batch Inference); batch is the sibling for deferred volume.

Shard the monorepo for compile-safe jobs

# shard_repo.py — illustrative package-level shards
from pathlib import Path
import json

ROOT = Path("packages")
shards = []
for pkg in sorted(ROOT.iterdir()):
    if not (pkg / "package.json").exists():
        continue
    files = [str(p) for p in pkg.rglob("*.ts") if "node_modules" not in p.parts]
    # Cap shard size so one poison file cannot sink a 2k-file job
    for i in range(0, len(files), 40):
        shards.append({"package": pkg.name, "files": files[i : i + 40]})

Path("batch/shards.json").write_text(json.dumps(shards, indent=2))
print(f"shards={len(shards)}")

✅ Shards align with package boundaries and TypeScript project references.
❌ One JSONL line per whole repository “to save orchestration.”

Build Bedrock batch JSONL with structured outputs

Force the model to emit a machine-checkable shape — patch intents, not prose:

{"recordId": "payments-0017", "modelInput": {"anthropic_version": "bedrock-2023-05-31", "max_tokens": 4096, "temperature": 0, "system": "Emit ONLY JSON {\"edits\":[{\"path\":\"\",\"unified_diff\":\"\"}]} for migrating aws-sdk v2 DocumentClient to @aws-sdk/lib-dynamodb. No commentary.", "messages": [{"role": "user", "content": "FILE path=packages/payments/src/repo.ts\n```\n...source...\n```"}]}}

# Illustrative — create batch inference job (CLI shape varies by model provider APIs)
aws s3 cp batch/input.jsonl s3://ai-batch-prod/refactors/2026-09-11/input.jsonl
aws bedrock create-model-invocation-job \
  --job-name refactor-ddb-client-20260911 \
  --role-arn arn:aws:iam::123456789012:role/BedrockBatchRole \
  --model-id anthropic.claude-sonnet-4-20250514-v1:0 \
  --input-data-config dataSource=s3InputDataConfig={s3Uri=s3://ai-batch-prod/refactors/2026-09-11/input.jsonl} \
  --output-data-config s3OutputDataConfig={s3Uri=s3://ai-batch-prod/refactors/2026-09-11/out/} \
  --timeout-duration-in-hours 24

IAM for the batch role: s3:GetObject/PutObject on those prefixes, bedrock:InvokeModel for the pinned model — nothing else. Sandbox the apply step like LLM Coding Agents on AWS: Safe Tool Sandboxes with Lambda.

Validate offline before any PR opens

#!/usr/bin/env bash
# apply_and_gate.sh — illustrative
set -euo pipefail
SHARD="$1"
git worktree add "/tmp/wt-$SHARD" -b "ai/refactor-$SHARD"
python3 tools/apply_batch_edits.py --shard "$SHARD" --worktree "/tmp/wt-$SHARD"
cd "/tmp/wt-$SHARD"
pnpm exec tsc -b --pretty false
pnpm --filter "./packages/${SHARD%%-*}" test -- --reporter=dot
# Only then:
# gh pr create --draft --title "refactor($SHARD): DocumentClient migration" ...

Reject shards that fail parse, typecheck, or tests. Quarantine them for interactive follow-up — do not widen types to force green.

Stacked PRs and human review

Mass merges hide regressions. Prefer:

1. Draft PRs per package shard (~100–300 lines changed)
2. CODEOWNERS must approve
3. Merge train: leaf packages first, apps last
4. Feature flag or dual-run when behavior might drift

main
  └─ ai/refactor-contracts-01   (merge first)
      └─ ai/refactor-payments-01
          └─ ai/refactor-web-01

For review quality, feed bots diff-scoped context only — same idea as semantic review later in this series and AI Code Review Bots.

Closing checklist

✅ Dos
– ✅ Pin model ID + prompt version in the job name and output metadata
– ✅ Shard by package; gate with tsc and unit tests in disposable worktrees
– ✅ Open stacked draft PRs; require CODEOWNERS
– ✅ Meter batch spend separately from interactive IDE usage
– ✅ Keep a reject queue for human/agent interactive repair

❌ Don’ts
– ❌ Don’t auto-merge thousands of AI lines overnight
– ❌ Don’t use free-form chat completions without a JSON schema gate
– ❌ Don’t give the batch role account-wide S3 or IAM mutate rights
– ❌ Don’t mix behavioral redesign into a “mechanical” batch prompt
– ❌ Don’t skip rollback tags on the pre-refactor SHA

Related reading

• Bedrock Prompt Caching and Batch Inference: Cut Latency and Cost
• LLM Coding Agents on AWS: Safe Tool Sandboxes with Lambda
• AI Code Review Bots: IAM, Secrets, and Least-Privilege Pipelines
• Cursor Rules for TypeScript Monorepos: Make AI Edits Stick

Last updated on September 11, 2026