7 views
A 10% canary that nobody rolls back is theater. Codify alias traffic shifting so CloudWatch alarms actively shift weight back to the last known good version — not a Slack message hoping a human is awake.
⚡ TL;DR: Publish versions; shift via alias weights or CodeDeploy linear/canary hooks; alarm on errors + p99; hook rollback to restore 100% to previous version automatically. Pair with Lambda Destinations and Lambda Warm Pools.
Lambda Alias Traffic Shifting: production guidance
Calling$LATEST from API Gateway or EventBridge means every publish is a hard cutover. Manual update-alias without alarms means a bad canary sits at 10% until someone notices. The durable pattern is: immutable versions + alias + automated rollback.
Alias weights and CodeDeploy
typescript
Manual weight sketch (without CodeDeploy):
// cdk with CodeDeploy canary
import * as codedeploy from "aws-cdk-lib/aws-codedeploy";
import * as lambda from "aws-cdk-lib/aws-lambda";
const fn = new lambda.Function(this, "Api", { /* ... */ });
const alias = new lambda.Alias(this, "Live", {
aliasName: "live",
version: fn.currentVersion,
});
new codedeploy.LambdaDeploymentGroup(this, "Deploy", {
alias,
deploymentConfig: codedeploy.LambdaDeploymentConfig.CANARY_10PERCENT_5MINUTES,
alarms: [
errorRateAlarm,
p99LatencyAlarm,
],
autoRollback: {
deploymentInAlarm: true,
failedDeployment: true,
stoppedDeployment: true,
},
});
bash
Prefer CodeDeploy configs you can name in PRs (# Shift 10% to new version 42, 90% on 41
aws lambda update-alias --function-name api --name live \
--routing-config AdditionalVersionWeights={"42"=0.1} \
--function-version 41
CANARY_10PERCENT_5MINUTES, LINEAR_10PERCENT_EVERY_1MINUTE) over bespoke cron that forgets to finish the shift.
Alarms that actually fire rollback
typescript
Also alarm on:
const errorRateAlarm = new cloudwatch.Alarm(this, "AliasErrors", {
metric: fn.metricErrors({
dimensionsMap: { FunctionName: fn.functionName, Resource: `${fn.functionName}:live` },
period: Duration.minutes(1),
statistic: "sum",
}),
threshold: 5,
evaluationPeriods: 3,
datapointsToAlarm: 2,
treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
});
- Downstream throttle spikes after deploy
- Cold-start INIT regressions if provisioned concurrency expected
- Iterator age for stream consumers behind the alias
- Duration p99 vs previous version baseline (anomaly or static)
Errors = 0 during canary because traffic was too low to see the bug — gate on minimum request count.
typescript
Use the traffic gate in pre/post hooks rather than as a silent success signal.
// Composite: errors OR (latency AND request count gate)
const trafficGate = new cloudwatch.Alarm(this, "MinTraffic", {
metric: fn.metricInvocations({ period: Duration.minutes(1), statistic: "sum" }),
threshold: 50,
comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD,
evaluationPeriods: 1,
treatMissingData: cloudwatch.TreatMissingData.BREACHING,
});
Pre-traffic and post-traffic hooks
python
Smoke fixtures should be schema-locked — see Lambda test-events-as-contracts patterns — so canaries exercise production shapes, not toy payloads.
# codedeploy hooks (illustrative)
def pre_traffic(event, context):
# smoke invoke new version with synthetic fixtures
invoke(version=event["TargetVersion"], payload=SMOKE)
put_lifecycle_success(event)
def post_traffic(event, context):
# compare error ratio new vs old over window
if error_ratio("new") > 2 * max(error_ratio("old"), 0.001):
put_lifecycle_failure(event)
else:
put_lifecycle_success(event)
Provisioned concurrency on the alias
Shift traffic without warming the new version and you page yourself with INIT spikes that look like bad code. Attach provisioned concurrency to the alias, and ensure the new version is prepared before weight moves. Pair with Lambda Warm Pools for agent-tool backends.Bake time and version retention
| Phase | Action |
|---|---|
| Canary | 5–15 min with alarms armed |
| Bake | Keep previous version invokable ≥1 hour |
| GC | Retain last N versions; never delete the rollback target mid-incident |
Observability during the shift
Emit version-dimensioned metrics for the canary window:Errors, Duration, Throttles, and a custom BusinessError count if your handler distinguishes 4xx-like outcomes. Dashboard the old vs new version side-by-side for the full bake period so on-call can see why auto-rollback fired without digging through CodeDeploy events.
IAM and blast radius
The CodeDeploy service role should only update the target alias and publish lifecycle events—not rewrite unrelated functions. Scope CloudWatch alarm actions tightly. If you use a custom rollback Lambda, give itlambda:UpdateAlias on one ARN and nothing else; a wildcards-happy rollback bot is how a canary failure becomes an account-wide outage.
Closing checklist
✅ Dos – ✅ Version every production deploy; clients call aliases only – ✅ Canary/linear with autoRollback on alarms – ✅ Synthetic smoke in pre-traffic hooks – ✅ Minimum-traffic gate before “success” – ✅ Keep previous version alive until bake time ends – ✅ Provisioned concurrency on the alias when latency-sensitive – ✅ Alarm on errors, p99, throttles, and iterator age ❌ Don’ts – ❌ Don’t publish$LATEST to callers
– ❌ Don’t canary without alarms wired to rollback
– ❌ Don’t ignore provisioned concurrency on the alias
– ❌ Don’t shift 100% on Friday without bake windows
– ❌ Don’t declare success on zero errors with near-zero traffic
Related reading
- Lambda Concurrency Provisioned SnapStart Reserved
- Lambda Destinations Async Failure Routing
- Lambda Warm Pools
- AWS Lambda Best Practices
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
