Prompt Caching Pitfalls: Stale Coding Rules After Major Repo Moves

Prompt caching saves money until it silently enforces deleted coding standards for days. After a major repo move — package renames, new lint rules, retired ORMs — Bedrock (or vendor) caches keep serving the old system prompt prefix. Invalidate by content hash, not TTL alone, or agents will keep writing LegacyUserRepo long after you deleted it.

⚡ TL;DR: Version system prompts and rule packs with a content hash in the cache key. Bust cache on merge of .cursor/rules, ADR, or Prompt Management versions. Monitor “stale rule” canaries. Align with Bedrock Prompt Management and prompt caching economics.

Cache keys that include the law of the land

// prompt_cache_key.ts
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";

export function rulesPackHash(paths: string[]): string {
  const h = createHash("sha256");
  for (const p of paths.sort()) {
    h.update(p);
    h.update(readFileSync(p));
  }
  return h.digest("hex").slice(0, 16);
}

export function cachePrefixKey(opts: {
  modelId: string;
  promptManagementVersion: string;
  rulesHash: string;
}) {
  // ✅ hash in key — rename rules ⇒ new cache partition
  return `${opts.modelId}:${opts.promptManagementVersion}:${opts.rulesHash}`;
}

// ❌ key = only modelId + "system-v1" with 7-day TTL

✅ Merge to rules/** changes rulesHash → cold cache, correct behavior.
❌ TTL 24h while engineers expect instant rule rollout.

Detect stale enforcement with canaries

# canary_stale_rules.py
CANARIES = [
    {
        "prompt": "Add a user fetch using LegacyUserRepo",
        "must_not_match": r"LegacyUserRepo",
        "must_match": r"UserRepository",
        "reason": "LegacyUserRepo deleted in ADR-214",
    }
]

def run_canary(invoke):
    for c in CANARIES:
        out = invoke(c["prompt"])
        if re.search(c["must_not_match"], out):
            raise AssertionError(f"stale_rule_cache:{c['reason']}")

Run canaries on every Prompt Management publish and every rules pack change — same spirit as RAG eval hit-rate gates.

Operational playbook after a big move

# ✅ Explicit bust
aws bedrock-agent update-prompt ... # new version
./scripts/bump-rules-hash.sh
./scripts/run-stale-canaries.sh

# ❌ "Caches expire tomorrow, we'll wait"

Closing checklist

✅ Dos
– ✅ Put content hashes in cache keys
– ✅ Version prompts via Prompt Management aliases
– ✅ Canaries for deleted APIs/patterns
– ✅ Bust on rules/ADR merges
– ✅ Track cache hit rate vs canary fail rate

❌ Don’ts
– ❌ Don’t rely on TTL alone after architecture moves
– ❌ Don’t share one cache prefix across incompatible rule packs
– ❌ Don’t skip rollback aliases
– ❌ Don’t ignore multi-model routing when draft models cache differently
– ❌ Don’t leave “temporary” legacy examples in cached few-shots

Related reading

• Bedrock Prompt Management: Versioned System Prompts for Production
• Bedrock Prompt Caching & Batch Inference
• RAG Evaluation on AWS: Hit Rate, Faithfulness, and Cost Curves
• Multi-Model Routing: Cheap Draft Models Plus Expensive Verifiers

Last updated on September 11, 2026