DynamoDB Streams Outbox: Domain Events Without Dual-Write Failures

DynamoDB Streams Outbox: Domain Events Without Dual-Write Failures

Writing a row to DynamoDB and separately publishing to EventBridge/SNS is a dual-write: either hop can succeed alone. The transactional outbox stores the domain event in the same TransactWriteItems as the business mutation; DynamoDB Streams (+ Lambda) relays events to the bus. Consumers see at-least-once delivery—you still need idempotency.

⚡ TL;DR: Put entity + outbox item in one transaction; stream the outbox (or entity with event attributes); publish from a Lambda with idempotent bus puts; delete/mark outbox after success. Pair with Lambda Powertools Idempotency and Event-Driven Architecture.

Same-transaction outbox item

# services/orders.py
import os, time, uuid
import boto3

ddb = boto3.client("dynamodb")
TABLE = os.environ["TABLE"]

def place_order(order_id: str, user_id: str, total_cents: int) -> None:
    event_id = str(uuid.uuid4())
    sk_event = f"OUTBOX#{event_id}"
    ddb.transact_write_items(
        TransactItems=[
            {
                "Put": {
                    "TableName": TABLE,
                    "Item": {
                        "pk": {"S": f"ORDER#{order_id}"},
                        "sk": {"S": "META"},
                        "user_id": {"S": user_id},
                        "total_cents": {"N": str(total_cents)},
                        "status": {"S": "PLACED"},
                    },
                    "ConditionExpression": "attribute_not_exists(pk)",
                }
            },
            {
                "Put": {
                    "TableName": TABLE,
                    "Item": {
                        "pk": {"S": f"ORDER#{order_id}"},
                        "sk": {"S": sk_event},
                        "type": {"S": "order.placed"},
                        "payload": {"S": f'{{"order_id":"{order_id}","total_cents":{total_cents}}}'},
                        "ts": {"N": str(int(time.time()))},
                    },
                }
            },
        ]
    )
# ❌ Dual write — EventBridge can succeed while DDB fails (or vice versa)
ddb.put_item(...)
events.put_events(...)

Streams relay

Filter the stream to outbox SK prefixes. Publisher Lambda uses partial batch failure for poison events (Lambda Event Source Mapping).

# relay/handler.py
import json, os, boto3

eb = boto3.client("events")
BUS = os.environ["BUS"]

def handler(event, context):
    failures = []
    for rec in event["Records"]:
        if rec["eventName"] not in ("INSERT", "MODIFY"):
            continue
        sk = rec["dynamodb"]["NewImage"]["sk"]["S"]
        if not sk.startswith("OUTBOX#"):
            continue
        try:
            img = rec["dynamodb"]["NewImage"]
            eb.put_events(Entries=[{
                "EventBusName": BUS,
                "Source": "orders",
                "DetailType": img["type"]["S"],
                "Detail": img["payload"]["S"],
                "Resources": [img["pk"]["S"]],
            }])
        except Exception:
            failures.append({"itemIdentifier": rec["eventID"]})
    return {"batchItemFailures": failures}

Idempotency and cleanup

  • Event id = outbox event_id; consumers dedupe.
  • Optionally mark published=true with a condition, or TTL the outbox item after success.
  • Never use Streams as your only audit log without a backup.

Closing checklist

✅ Dos
– ✅ Mutate entity + outbox in one TransactWriteItems
– ✅ Relay via Streams with partial batch failure
– ✅ Give every event a stable id for consumer idempotency
– ✅ TTL or mark outbox rows after publish
– ✅ Alarm on relay iterator age / failures

❌ Don’ts
– ❌ Don’t PutItem then PutEvents as two calls
– ❌ Don’t assume Streams delivery is exactly-once
– ❌ Don’t publish huge payloads inline—use pointers to S3 when needed
– ❌ Don’t skip conditions on the entity put (lost updates)
– ❌ Don’t let relay concurrency overwhelm the bus (reserved concurrency)

Related reading

Last updated on September 11, 2026


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

1 Comment

Leave a Reply