Idempotency Keys End-to-End: API Gateway Through Step Functions Safely

Idempotency Keys End-to-End: API Gateway Through Step Functions Safely
8 views
Clients retry on timeouts. API Gateway retries. Step Functions retries. Lambda retries. Without a single idempotency key that travels the whole path, you double-charge and double-provision. Treat the key as part of the API contract: required header, validated format, stored with request hash, and reused as the Step Functions execution name or business key.
⚡ TL;DR: Require Idempotency-Key at the edge; map it into the state machine input; use it as name for StartExecution (or a DynamoDB idempotency record); make every side-effecting task Powertools-idempotent. See Lambda Powertools Idempotency and Lambda Durable Patterns.

Idempotency Keys End-to-End: production guidance

http
POST /v1/orders HTTP/1.1
Idempotency-Key: 018f3c2a-9c1e-7b2d-a111-deadbeefcafe
Content-Type: application/json

{"sku":"sku-1","qty":2}
python
# api/handler.py
import hashlib, json, os, re
import boto3
from botocore.exceptions import ClientError

sfn = boto3.client("stepfunctions")
KEY_RE = re.compile(r"^[A-Za-z0-9._-]{8,128}$")
SM_ARN = os.environ["SM_ARN"]

def handler(event, _):
 key = event["headers"].get("idempotency-key") or event["headers"].get("Idempotency-Key")
 if not key or not KEY_RE.match(key):
 return {"statusCode": 400, "body": '{"error":"idempotency_key_required"}'}
 body = event.get("body") or "{}"
 payload = {
 "idempotency_key": key,
 "body_hash": hashlib.sha256(body.encode()).hexdigest(),
 "body": json.loads(body),
 }
 try:
 # ✅ Execution name enforces uniqueness per state machine
 sfn.start_execution(
 stateMachineArn=SM_ARN,
 name=key,
 input=json.dumps(payload),
 )
 except ClientError as e:
 if e.response["Error"]["Code"] != "ExecutionAlreadyExists":
 raise
 # same key → return prior status (fetch describe_execution)
 return {"statusCode": 202, "body": json.dumps({"id": key})}

Conflict when body changes

If the same key arrives with a different body hash, return 409 Conflict. Store (key → hash) in DynamoDB when execution names are insufficient (e.g., express workflows or reuse after retention).
python
# ❌ Blind StartExecution without hashing body
sfn.start_execution(stateMachineArn=SM_ARN, input=body)

Task-level idempotency inside the flow

Map $.idempotency_key into each Lambda. Compose with a task token suffix for fan-out:
python
from aws_lambda_powertools.utilities.idempotency import (
 IdempotencyConfig, DynamoDBPersistenceLayer, idempotent_function,
)

persistence = DynamoDBPersistenceLayer(table_name=os.environ["IDEMP_TABLE"])
config = IdempotencyConfig(event_key_jmespath="idempotency_key", expires_after_seconds=86400)

@idempotent_function(data_keyword_argument="event", config=config, persistence_store=persistence)
def charge(event: dict) -> dict:
 return payment_provider.charge(event["body"])

Timeouts and client behavior

Document: clients must reuse the key on retry; 202 means “accepted,” poll status by key; do not mint a new key for the same user intent. Align API Gateway integration timeout with Step Functions async start (do not wait for the whole workflow synchronously).

Closing checklist

✅ Dos – ✅ Require and validate Idempotency-Key at API Gateway/Lambda – ✅ Use key as Step Functions execution name or durable DDB record – ✅ Hash body; conflict on mismatch – ✅ Idempotent every side-effecting task – ✅ Return stable status for duplicate submissions ❌ Don’ts – ❌ Don’t start executions with random names for retried intents – ❌ Don’t wait synchronously for long workflows behind API Gateway – ❌ Don’t expire idempotency records shorter than max client retry window – ❌ Don’t accept empty or ultra-short keys – ❌ Don’t assume Step Functions retries alone make tasks safe

Related reading

Last updated on September 11, 2026 Related: AWS Multi-Tenant SaaS Control Plane Architecture: API Gateway, ECS, DynamoDB, Step Functions, and EventBridge; Lambda Function URLs with Streaming: Replace API Gateway for 90% of Use Cases; Step Functions Express Workflows: Orchestrate Lambda at 100K Events Per Second.

Watch: 100 Days of AI on YouTube

Short videos from CheatCoders, one AI engineering topic per day.

Open the playlist on YouTube · CheatCoders on YouTube

Deep-dive PDF

Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.

Already subscribed? or open the subscribe page.


Discover more from CheatCoders

Subscribe to get the latest posts sent to your email.

1 Comment

Leave a comment

No account needed. Name and email are optional.