Day 66: Audit Trails Humans Can Replay

“The agent did something weird” is not a postmortem. You need deterministic session logs — prompts (redacted), tool calls, artifacts hashes, model IDs — that a human can replay without guessing.

⚡ TL;DR: Append-only event log per run_id. Store artifact hashes, not only chat. Replay tool stubs in a dry-run harness.

Event schema

{
  "run_id": "run_7f3a",
  "seq": 12,
  "type": "tool_call",
  "tool": "apply_intent",
  "args_hash": "…",
  "result_hash": "…",
  "model": "anthropic.claude-…",
  "ts": "2026-09-11T15:01:02Z"
}

# audit/log.py
import json, hashlib
from pathlib import Path

def append_event(run_dir: Path, event: dict):
    event = dict(event)
    event["event_hash"] = hashlib.sha256(
        json.dumps(event, sort_keys=True).encode()
    ).hexdigest()
    with (run_dir / "events.jsonl").open("a") as f:
        f.write(json.dumps(event) + "\n")  # ✅ append-only

Replay

python audit/replay.py --run artifacts/run_7f3a --dry-run

Replay should re-validate hashes and re-execute pure tools; impure tools stay stubbed with recorded results.

Closing checklist

• [ ] JSONL append-only events
• [ ] Hash args/results
• [ ] Record model + prompt version
• [ ] Redact secrets/PII in logs
• [ ] Human replay dry-run tool

Series navigation

Day 65: PII Redaction Before Embeddings · Day 67: Supply Chain: Model and Image Provenance

Last updated September 11, 2026