Idempotency Keys End-to-End: API Gateway Through Step Functions Safely

Clients retry on timeouts. API Gateway retries. Step Functions retries. Lambda retries. Without a single idempotency key that travels the whole path, you double-charge and double-provision. Treat the key as part of the API contract: required header, validated format, stored with request hash, and reused as the Step Functions execution name or business key.

⚡ TL;DR: Require Idempotency-Key at the edge; map it into the state machine input; use it as name for StartExecution (or a DynamoDB idempotency record); make every side-effecting task Powertools-idempotent. See Lambda Powertools Idempotency and Lambda Durable Patterns.

Edge contract

POST /v1/orders HTTP/1.1
Idempotency-Key: 018f3c2a-9c1e-7b2d-a111-deadbeefcafe
Content-Type: application/json

{"sku":"sku-1","qty":2}

# api/handler.py
import hashlib, json, os, re
import boto3
from botocore.exceptions import ClientError

sfn = boto3.client("stepfunctions")
KEY_RE = re.compile(r"^[A-Za-z0-9._-]{8,128}$")
SM_ARN = os.environ["SM_ARN"]

def handler(event, _):
    key = event["headers"].get("idempotency-key") or event["headers"].get("Idempotency-Key")
    if not key or not KEY_RE.match(key):
        return {"statusCode": 400, "body": '{"error":"idempotency_key_required"}'}
    body = event.get("body") or "{}"
    payload = {
        "idempotency_key": key,
        "body_hash": hashlib.sha256(body.encode()).hexdigest(),
        "body": json.loads(body),
    }
    try:
        # ✅ Execution name enforces uniqueness per state machine
        sfn.start_execution(
            stateMachineArn=SM_ARN,
            name=key,
            input=json.dumps(payload),
        )
    except ClientError as e:
        if e.response["Error"]["Code"] != "ExecutionAlreadyExists":
            raise
        # same key → return prior status (fetch describe_execution)
    return {"statusCode": 202, "body": json.dumps({"id": key})}

Conflict when body changes

If the same key arrives with a different body hash, return 409 Conflict. Store (key → hash) in DynamoDB when execution names are insufficient (e.g., express workflows or reuse after retention).

# ❌ Blind StartExecution without hashing body
sfn.start_execution(stateMachineArn=SM_ARN, input=body)

Task-level idempotency inside the flow

Map $.idempotency_key into each Lambda. Compose with a task token suffix for fan-out:

from aws_lambda_powertools.utilities.idempotency import (
    IdempotencyConfig, DynamoDBPersistenceLayer, idempotent_function,
)

persistence = DynamoDBPersistenceLayer(table_name=os.environ["IDEMP_TABLE"])
config = IdempotencyConfig(event_key_jmespath="idempotency_key", expires_after_seconds=86400)

@idempotent_function(data_keyword_argument="event", config=config, persistence_store=persistence)
def charge(event: dict) -> dict:
    return payment_provider.charge(event["body"])

Timeouts and client behavior

Document: clients must reuse the key on retry; 202 means “accepted,” poll status by key; do not mint a new key for the same user intent. Align API Gateway integration timeout with Step Functions async start (do not wait for the whole workflow synchronously).

Closing checklist

✅ Dos
– ✅ Require and validate Idempotency-Key at API Gateway/Lambda
– ✅ Use key as Step Functions execution name or durable DDB record
– ✅ Hash body; conflict on mismatch
– ✅ Idempotent every side-effecting task
– ✅ Return stable status for duplicate submissions

❌ Don’ts
– ❌ Don’t start executions with random names for retried intents
– ❌ Don’t wait synchronously for long workflows behind API Gateway
– ❌ Don’t expire idempotency records shorter than max client retry window
– ❌ Don’t accept empty or ultra-short keys
– ❌ Don’t assume Step Functions retries alone make tasks safe

Related reading

• Lambda Powertools Idempotency: DynamoDB Keys That Survive Retries
• Lambda Durable Patterns: Avoid Step Functions Until Complexity Earns It
• Lambda Test Events as Contracts: Schema-Locked Fixtures Across Teams
• Lambda Recursive Loop Detection: Break Accidental Invoke Storms Early

Last updated on September 11, 2026