12 views
This guide focuses on AI audit trails for production systems, with practical trade-offs for reliability, security, and cost.
“The agent did something weird” is not a postmortem. You need deterministic session logs — prompts (redacted), tool calls, artifacts hashes, model IDs — that a human can replay without guessing.
⚡ TL;DR: Append-only event log per run_id. Store artifact hashes, not only chat. Replay tool stubs in a dry-run harness.
AI audit trails: production guidance
json
{
"run_id": "run_7f3a",
"seq": 12,
"type": "tool_call",
"tool": "apply_intent",
"args_hash": "…",
"result_hash": "…",
"model": "anthropic.claude-…",
"ts": "2026-09-11T15:01:02Z"
}
python
# audit/log.py
import json, hashlib
from pathlib import Path
def append_event(run_dir: Path, event: dict):
event = dict(event)
event["event_hash"] = hashlib.sha256(
json.dumps(event, sort_keys=True).encode()
).hexdigest()
with (run_dir / "events.jsonl").open("a") as f:
f.write(json.dumps(event) + "\n") # ✅ append-only
Replay
bash
Replay should re-validate hashes and re-execute pure tools; impure tools stay stubbed with recorded results.
python audit/replay.py --run artifacts/run_7f3a --dry-run
Closing checklist
- [ ] JSONL append-only events
- [ ] Hash args/results
- [ ] Record model + prompt version
- [ ] Redact secrets/PII in logs
- [ ] Human replay dry-run tool
Series navigation
Day 65: PII Redaction Before Embeddings · Day 67: Supply Chain: Model and Image Provenance Last updated September 11, 2026 Related: Post-Incident AI Summaries: Ground Drafts in Timeline Artifacts Humans Edit; Day 76: p99 of Agents: Queueing, Not Just Model Latency; Day 77: Caching RAG Answers Safely.Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
