Day 72: Prompt Caching Pitfalls

Prompt caching saves money until it serves yesterday’s security rules after a repo move. Cache keys must include anything that changes behavior: git SHA of rules, tool schema hash, guardrail version.

⚡ TL;DR: Include rules_sha, tools_sha, guardrail_ver in cache keys. Invalidate on move/rename of policy packs. Prefer explicit Bedrock prompt caching controls over accidental sticky contexts.

Cache key design

import hashlib

def cache_key(rules_dir: str, tools_schema: str, guardrail_ver: str, prefix: str) -> str:
    h = hashlib.sha256()
    h.update(git_tree_sha(rules_dir).encode())
    h.update(tools_schema.encode())
    h.update(guardrail_ver.encode())
    h.update(prefix.encode())
    return h.hexdigest()

// ❌ BAD: cache only on "system prompt text" while rules are loaded via tool later
// ✅ GOOD: hash the rules pack at the SHA the agent checked out

Pitfalls

Closing checklist

• [ ] Hash rules + tools + guardrails into keys
• [ ] Tenant-isolate caches
• [ ] Invalidate on pack moves
• [ ] Metric cache hit rate vs incident rate
• [ ] Document what is safe to cache

Series navigation

Day 71: Token Budgets per PR and per Engineer · Day 73: Draft-Then-Verify Routing

Last updated September 11, 2026