34 views
Redis Redlock looks like mutual exclusion until a paused GC, NTP jump, or minority quorum hands two clients the same “lock.” The unfair advantage is treating correctness as fencing tokens + conditional writes on the system of record—not as a TTL flag in a cache.
⚡ TL;DR: Use Redis locks only for best-effort coordination; for money/inventory use DynamoDB conditional puts or Postgres FOR UPDATE with version tokens; always check a monotonic fence on every write. Pair with Lease-Based ECS Leadership and Exactly-Once Illusions.
Distributed Locks Reality Check: production guidance
text
Clock skew between Redis nodes makes multi-key Redlock worse, not better, under partitions.
Client A acquires lock (TTL 30s)
Client A pauses 40s (GC / laptop sleep / SIGSTOP)
TTL expires → Client B acquires lock
Client A resumes and writes anyway ← double writer
| Mechanism | Safety under pause | Good for |
|---|---|---|
| Redis SET NX + TTL | Weak | Soft dedupe, cache stampede |
| Redlock (multi-node) | Still weak | Same — not safety-critical |
| DynamoDB conditional + version | Strong | Inventory, payments |
| DB row lock + fence | Strong | Single-region source of truth |
Fencing tokens that actually work
python
# ✅ DynamoDB: lock row + monotonic fence; writers must present fence
import boto3
from botocore.exceptions import ClientError
ddb = boto3.resource("dynamodb").Table("leases")
def acquire(resource_id: str, owner: str, ttl_epoch: int) -> int:
# Returns fencing token (version)
try:
out = ddb.update_item(
Key={"pk": resource_id},
UpdateExpression="SET #o=:o, #t=:t, #v=if_not_exists(#v,:z)+ :one",
ConditionExpression="attribute_not_exists(pk) OR #t < :now",
ExpressionAttributeNames={"#o": "owner", "#t": "ttl", "#v": "fence"},
ExpressionAttributeValues={
":o": owner, ":t": ttl_epoch, ":now": ttl_epoch - 1,
":z": 0, ":one": 1,
},
ReturnValues="ALL_NEW",
)
return int(out["Attributes"]["fence"])
except ClientError as e:
if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
raise RuntimeError("lock held") from e
raise
def write_with_fence(item_table, pk: str, fence: int, body: dict) -> None:
# ✅ reject stale owners even if they still think they hold Redis lock
item_table.put_item(
Item={**body, "pk": pk, "fence": fence},
ConditionExpression="attribute_not_exists(pk) OR fence < :f",
ExpressionAttributeValues={":f": fence},
)
python
# ❌ Classic Redlock-only “safety”
# redis.set(name, owner, nx=True, ex=30)
# do_payment() # no fence on the ledger write
# redis.delete(name)
When a soft lock is fine
Cache rebuilds, single-flight computations, and “only one metrics scraper” can use Redis NX+TTL. Document that a double run is harmless. If double run is not harmless, you do not have a soft lock problem—you have a storage correctness problem.Closing checklist
✅ Dos – ✅ Put fencing tokens on every authoritative write – ✅ Prefer conditional writes on DynamoDB/Postgres – ✅ Treat Redis locks as hints / single-flight only – ✅ Bound critical sections; renew leases explicitly – ✅ Chaos-test process pauses longer than TTL ❌ Don’ts – ❌ Don’t use Redlock for payments or inventory alone – ❌ Don’t trust wall-clock TTL under GC pauses – ❌ Don’t delete locks without ownership check (GET then DEL race)
– ❌ Don’t stretch lock TTL to “be safe” without fencing
Related reading
- Lease-Based ECS Leadership: Singleton Cron That Survives Rolling Deploys
- Exactly-Once Illusions: Design At-Least-Once Plus Truly Idempotent Handlers
- Saga Compensations Under Partial Failure
- Idempotency Keys End-to-End
Watch: 100 Days of AI on YouTube
Short videos from CheatCoders, one AI engineering topic per day.
Most viewed
- Python Decorators Explained: From Simple Wrappers to Production Patterns
- Claude Code vs Cursor vs GitHub Copilot: Choose by Workflow, and Set Up One Repo That Works With All Three
- AI Agent Frameworks in 2025: LangGraph vs CrewAI vs AutoGen vs Raw API
- Python String Methods: Every str Method With Real Production Examples
- Css Animation Performance: Performance-Optimized Motion for Modern UIs
Newly added
- AWS Systems Manager Parameter Store: Hierarchical Runtime Config for Coding Agents
- Amazon Cognito Identity Pools: Federate Coding-Agent Tool Callers Without Long-Lived Keys
- Amazon Athena for Coding Agents: Query Tool Audit Logs and Eval Traces on S3 Without a Warehouse
- AWS Batch for Coding Agents: Parallel Overnight PR Reviews and Long-Running Jobs Without Lambda Timeouts
- Amazon Bedrock Prompt Management: Version and A/B Test Coding-Agent System Prompts
Deep-dive PDF
Get the expanded guide for this post — extra diagrams-style checklists, failure modes, and a production walkthrough. Free when you subscribe to CheatCoders.
Already subscribed? or open the subscribe page.
Discover more from CheatCoders
Subscribe to get the latest posts sent to your email.
