5 views
This guide focuses on Lambda SQS Partial Batch Failures for production systems, with practical trade-offs for reliability, security, and cost.
With classic SQS→Lambda, throwing on one bad record redrives the entire batch. Healthy messages get duplicate side effects; poison messages loop until the DLQ — or forever if you misconfigure maxReceiveCount. Partial batch failure reporting fixes the contract: return only the failing message IDs.
⚡ TL;DR: EnableReportBatchItemFailures; process records independently; return{ batchItemFailures: [{ itemIdentifier }] }; classify poison vs transient; DLQ after N receives; idempotent handlers. Pair with Lambda Event Source Mapping Parallelization and Lambda Timeouts & DLQs.
Lambda SQS Partial Batch Failures: production guidance
typescript
// sqsPartial.ts
import type { SQSEvent, SQSBatchResponse, SQSRecord } from "aws-lambda";
export async function handler(event: SQSEvent): Promise<SQSBatchResponse> {
const failures: { itemIdentifier: string }[] = [];
await Promise.all(
event.Records.map(async (rec) => {
try {
await processOne(rec);
} catch (err) {
console.error(JSON.stringify({
msg: "record_failed",
messageId: rec.messageId,
err: String(err),
}));
failures.push({ itemIdentifier: rec.messageId });
}
})
);
// ✅ Only failing IDs — successes are deleted by Lambda/SQS
return { batchItemFailures: failures };
}
async function processOne(rec: SQSRecord) {
const body = JSON.parse(rec.body);
if (!body.orderId) {
// Poison: validation error — still report failure so it counts toward redrive,
// or route to a quarantine queue explicitly if you prefer not to retry.
throw new Error("poison_missing_orderId");
}
await idempotentWrite(body); // must tolerate duplicates
}
IaC switch
typescript
fn.addEventSource(new lambdaEventSources.SqsEventSource(queue, {
batchSize: 10,
reportBatchItemFailures: true, // ✅ required
maxConcurrency: 40,
}));
yaml
# Queue redrive — poison lands here after N receives
RedrivePolicy:
deadLetterTargetArn: !GetAtt Dlq.Arn
maxReceiveCount: 5
Poison vs transient
| Error class | Action |
|---|---|
| Schema invalid / forever-fail | Fail item → DLQ after maxReceive; alert |
| Downstream 429 / timeout | Fail item → retry; backoff via visibility |
| Bug in code | Fail item → fix + replay from DLQ |
| Handler timeout mid-batch | Untouched records may redrive — keep batches small |
typescript
// ❌ Anti-pattern: catch-all swallow
try {
await processOne(rec);
} catch {
// silent — message deleted, data lost
}
Closing checklist
✅ Dos– ✅
ReportBatchItemFailures + return messageIds only– ✅ Idempotent writes keyed by business id
– ✅ DLQ with alarm on
ApproximateNumberOfMessagesVisible– ✅ Separate validation poisons from transient throttles in metrics
– ✅ Keep batch size modest relative to timeout ❌ Don’ts
– ❌ Don’t throw at the top level for a single bad record
– ❌ Don’t swallow errors (acks poison as success)
– ❌ Don’t set
maxReceiveCount to absurd values without a DLQ– ❌ Don’t process the batch as one transaction unless you mean it
Related reading
- Lambda Event Source Mapping Parallelization
- Lambda Timeouts, Retries, and DLQs
- Lambda Destinations
- Lambda Kinesis Bisect on Error
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.

Pingback: Lambda Kinesis Bisect on Error: Isolate Bad Records Without Lag - CheatCoders