8 views
A 99.9% monthly SLO is ~43 minutes of error budget. Teams that only page when the budget is gone already lost. The unfair advantage is budget-driven shedding: as burn rate spikes, disable non-critical features and reject low-priority traffic before the dependency meltdown becomes a hard outage.
⚡ TL;DR: Compute multi-window burn (1h/6h); at 2× budget burn shed batch/analytics; at 5× shed non-critical reads; protect auth/checkout until last; automate via AppConfig/feature flags. Pair with Multi-Tenant Rate Limits and Lambda Reserved Concurrency.
AI load shedding: production guidance
| Burn (vs budget) | Window | Action |
|---|---|---|
| 1× | 6h | Page SRE; freeze deploys |
| 2× | 1h | Shed exports, search fancy, AI extras |
| 5× | 5m | Shed non-critical GETs; keep pay/auth |
| 10× | 5m | Admit only health + checkout; open status page |
typescript
// ✅ Edge middleware: priority classes
type Priority = "critical" | "deferrable" | "batch";
const shedLevel = await appConfig.getShedLevel(); // 0..3 from burn alarms
export function admit(route: { priority: Priority }): boolean {
if (shedLevel >= 3) return route.priority === "critical";
if (shedLevel >= 2) return route.priority !== "batch";
if (shedLevel >= 1) return route.priority !== "batch" || Math.random() < 0.2;
return true;
}
Wire CloudWatch to AppConfig
bash
# ✅ Alarm on fast-burn composite → EventBridge → Lambda → AppConfig flag
# Metric math: error_rate / (1 - SLO) over 5m and 1h windows (Google multi-window)
python
❌ Only alerting humans with no automated shed — overnight pages burn the whole budget before anyone acks.
# ✅ Lambda updater (sketch)
import boto3
appconfig = boto3.client("appconfig")
def set_shed_level(level: int) -> None:
# write hosted config / flag variant; agents poll every 15s
assert 0 <= level <= 3
# ... StartDeployment of new config profile version
Protect the golden paths
Mark auth, session refresh, checkout, and payment webhooks ascritical. Everything else must declare a priority or default to deferrable. Load tests must prove shedding restores p99 for critical routes under 3× traffic.
Closing checklist
✅ Dos – ✅ Define multi-window burn alerts – ✅ Map burn → shed level in runbooks and code – ✅ Prefer 503 +Retry-After for shed traffic
– ✅ Freeze deploys automatically on budget burn
– ✅ Review shed events in weekly reliability meeting
❌ Don’ts
– ❌ Don’t shed auth/pay before vanity features
– ❌ Don’t use only latency pages without error budget context
– ❌ Don’t leave shed flags stuck on after recovery
– ❌ Don’t forget idempotency when clients retry after 503
Related reading
- Multi-Tenant Rate Limits
- Lambda Reserved Concurrency
- Continuous Path Verification: Minute Canaries
- API Gateway WebSocket Fan-Out: Coalesce Pushes
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
Pingback: Continuous Path Verification: Minute Canaries for Auth, Pay, and Search - CheatCoders