5 views
This guide focuses on AI feature SLOs for production systems, with practical trade-offs for reliability, security, and cost.
AI features need SLOs beyond HTTP 200. Availability, faithfulness, and latency compete — pick which ones page humans, and which only ticket.
⚡ TL;DR: Define SLIs: success rate, grounded-answer rate, p95 latency, budget burn. Page on availability + one quality signal. Don’t page on every eval dip.
AI feature SLOs: production guidance
| SLI | SLO | Alert |
|---|---|---|
| Gateway availability | 99.9% / 30d | Page |
| p95 E2E latency (interactive) | < 8s | Page if burn |
| Faithfulness (sampled) | ≥ 92% weekly | Ticket |
| Cost / PR | < $X squad | Ticket |
python
❌ Paging the on-call because offline NDCG moved 0.3 points overnight.
# ✅ Error budget for availability
def error_budget_remaining(uptime: float, slo=0.999, window_min=30*24*60):
allowed_down = (1 - slo) * window_min
used = (1 - uptime) * window_min
return allowed_down - used
Failure modes
Faithfulness jobs sampling only happy-path FAQ questions. Include adversarial and out-of-domain. Latency SLOs that include waiting on humans will never be met — exclude gate waits from the interactive SLO.Closing checklist
- [ ] SLIs documented per AI feature
- [ ] Page vs ticket explicitly chosen
- [ ] Faithfulness sampling job exists
- [ ] Latency excludes human-gate wait (Day 76)
- [ ] Review SLOs quarterly
Series navigation
Day 92: Threat Model an AI Feature · Day 94: Change Management for Prompts Last updated September 11, 2026 Related: Day 76: p99 of Agents: Queueing, Not Just Model Latency; Day 77: Caching RAG Answers Safely; Day 78: Load Shedding When the Model Is Sick.Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
