Lambda SQS Partial Batch Failures: Poison Messages Without Infinite Redrive

Lambda SQS Partial Batch Failures: Poison Messages Without Infinite Redrive
5 views
This guide focuses on Lambda SQS Partial Batch Failures for production systems, with practical trade-offs for reliability, security, and cost. With classic SQS→Lambda, throwing on one bad record redrives the entire batch. Healthy messages get duplicate side effects; poison messages loop until the DLQ — or forever if you misconfigure maxReceiveCount. Partial batch failure reporting fixes the contract: return only the failing message IDs.
⚡ TL;DR: Enable ReportBatchItemFailures; process records independently; return { batchItemFailures: [{ itemIdentifier }] }; classify poison vs transient; DLQ after N receives; idempotent handlers. Pair with Lambda Event Source Mapping Parallelization and Lambda Timeouts & DLQs.

Lambda SQS Partial Batch Failures: production guidance

typescript
// sqsPartial.ts
import type { SQSEvent, SQSBatchResponse, SQSRecord } from "aws-lambda";

export async function handler(event: SQSEvent): Promise<SQSBatchResponse> {
  const failures: { itemIdentifier: string }[] = [];

  await Promise.all(
    event.Records.map(async (rec) => {
      try {
        await processOne(rec);
      } catch (err) {
        console.error(JSON.stringify({
          msg: "record_failed",
          messageId: rec.messageId,
          err: String(err),
        }));
        failures.push({ itemIdentifier: rec.messageId });
      }
    })
  );

  // ✅ Only failing IDs — successes are deleted by Lambda/SQS
  return { batchItemFailures: failures };
}

async function processOne(rec: SQSRecord) {
  const body = JSON.parse(rec.body);
  if (!body.orderId) {
    // Poison: validation error — still report failure so it counts toward redrive,
    // or route to a quarantine queue explicitly if you prefer not to retry.
    throw new Error("poison_missing_orderId");
  }
  await idempotentWrite(body); // must tolerate duplicates
}

IaC switch

typescript
fn.addEventSource(new lambdaEventSources.SqsEventSource(queue, {
  batchSize: 10,
  reportBatchItemFailures: true, // ✅ required
  maxConcurrency: 40,
}));
yaml
# Queue redrive — poison lands here after N receives
RedrivePolicy:
  deadLetterTargetArn: !GetAtt Dlq.Arn
  maxReceiveCount: 5

Poison vs transient

Error class Action
Schema invalid / forever-fail Fail item → DLQ after maxReceive; alert
Downstream 429 / timeout Fail item → retry; backoff via visibility
Bug in code Fail item → fix + replay from DLQ
Handler timeout mid-batch Untouched records may redrive — keep batches small
typescript
// ❌ Anti-pattern: catch-all swallow
try {
  await processOne(rec);
} catch {
  // silent — message deleted, data lost
}

Closing checklist

✅ Dos
– ✅ ReportBatchItemFailures + return messageIds only
– ✅ Idempotent writes keyed by business id
– ✅ DLQ with alarm on ApproximateNumberOfMessagesVisible
– ✅ Separate validation poisons from transient throttles in metrics
– ✅ Keep batch size modest relative to timeout ❌ Don’ts
– ❌ Don’t throw at the top level for a single bad record
– ❌ Don’t swallow errors (acks poison as success)
– ❌ Don’t set maxReceiveCount to absurd values without a DLQ
– ❌ Don’t process the batch as one transaction unless you mean it

Related reading

Last updated on September 11, 2026 Related: SQS FIFO + Lambda: Ordered Agent Job Queues Without Double-Applies; Day 31: Lambda + Bedrock: Sync, Stream, and Batch; Hidden AWS Lambda Cold Start Fix — Zero Cost, Zero Latency.

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

1 Comment

Leave a comment

No account needed. Name and email are optional.