AI Incident Timeline: Build a Reliable Summarizer

AI incident timeline: production guidance

⚡ TL;DR: Ingest PagerDuty/Opsgenie, deploy markers, and chat exports. Draft a timeline with citations. Human edits in a doc. Only then allow “post to #incidents.” No silent auto-post.

Pipeline

Artifacts → normalize → retrieve → draft timeline → human edit → optional post

# ✅ Citation-forced draft
def draft_timeline(artifacts: list[dict]) -> dict:
    evidence = index(artifacts)
    draft = llm_draft(evidence)
    for claim in draft["claims"]:
        if claim["cite_id"] not in evidence:
            claim["status"] = "ungrounded"
    draft["claims"] = [c for c in draft["claims"] if c["status"] != "ungrounded"]
    draft["ready_to_post"] = False
    return draft

Acceptance criteria

• [ ] Every bullet cites an artifact id
• [ ] UI blocks post while ready_to_post is false
• [ ] Redacts customer names by default
• [ ] Stores draft + final under incident id
• [ ] Eval: replay 5 past SEVs; faithfulness score ≥ threshold

Failure modes

Closing checklist

• [ ] Artifact connectors with least privilege
• [ ] Human edit gate
• [ ] Redaction layer
• [ ] Faithfulness eval harness
• [ ] Runbook for summarizer outage (write manually)

Series navigation