Kafka to Lambda Backpressure: Lag SLOs That Protect Downstream Sinks

Kafka consumer lag is a feature until Lambda parallelization turns it into a thundering herd against DynamoDB, RDS, or a partner API. Backpressure means: define lag SLOs per topic, cap concurrent invokes, and shed or slow consumption when sink error rates or latency burn the budget—not when the pager already screams.

⚡ TL;DR: Set lag SLO (e.g., p99 lag < 30s); bound event-source parallelization / reserved concurrency; propagate sink 429/5xx into slower polls or fewer shards processed; never “fix” lag by maxing concurrency into a choking DB. Related: Lambda Event Source Mapping: Parallelization Factors That Match Sinks, Lambda Reserved Concurrency: Bulkheads, Lambda Response Streaming: Backpressure.

Lag as an SLO, not a vanity gauge

SLO: consumer_lag_seconds{topic="orders"} p99 < 30 over 5m
Error budget: burn alerts at 2x / 5x consumption rate

Lag growing while sink p95 is healthy may mean under-scaled consumers. Lag flat while sink 5xx climbs means you need less concurrency.

Cap the herd

For MSK/Kafka → Lambda (or Kafka → SQS → Lambda), set reserved concurrency to what the sink sustains in soak tests. Parallelization factor / per-partition concurrency must match write capacity.

# SAM sketch — concurrency bulkhead
OrderConsumer:
  Type: AWS::Serverless::Function
  Properties:
    ReservedConcurrentExecutions: 50
    Events:
      Kafka:
        Type: MSK
        Properties:
          Stream: !Ref OrdersCluster
          Topics:
            - orders
          BatchSize: 100

# ❌ Unbounded: scale Lambda to wipe lag at all costs
# ReservedConcurrentExecutions omitted + high parallelization

Adaptive throttle signal

Expose sink health to the consumer path:

# consumer/handler.py
class SinkBusy(Exception):
    pass

def handler(event, context):
    if sink_token_bucket.try_take(len(event["records"])) is False:
        # Return failures so batch retries later — or pause partition
        raise SinkBusy("dynamo_throttle")
    process_batch(event["records"])

Prefer partial batch failure where supported so healthy partitions proceed. For classic Kafka consumers on ECS, lower max.poll.records and pause partitions when circuit breakers open.

Protecting the sink

Closing checklist

✅ Dos
– ✅ Publish lag histograms and SLO burn alerts
– ✅ Soak-test sink capacity; set reserved concurrency from evidence
– ✅ Circuit-break when sink throttles
– ✅ Prefer partial failures over all-or-nothing batch retry
– ✅ Separate critical vs deferred topics

❌ Don’ts
– ❌ Don’t max parallelization to “clear lag” into RDS
– ❌ Don’t ignore 429s as blips
– ❌ Don’t use one shared concurrency pool for all tenants/topics
– ❌ Don’t alert only on absolute lag without rate of change
– ❌ Don’t skip DLQ/quarantine for poison payloads

Related reading

• Lambda Event Source Mapping: Parallelization Factors That Match Sinks
• Lambda Reserved Concurrency: Bulkheads That Protect Tenant Workloads
• Lambda Response Streaming: Backpressure When Clients Stall Mid-Transfer
• Lambda Kinesis Bisect on Error: Isolate Bad Records Without Lag

Last updated on September 11, 2026