MemoryDB vs ElastiCache for Coding-Agent Session State: Durable Cursors and Tool-Result Caches

A coding agent is twelve steps into a refactor. Its session state says which step it’s on, which files it has touched and the hash of the last test run. Then the cache node holding that state fails over, the keys come back empty, and the agent starts the task again from the top: it re-plans, re-reads the repo, re-applies a half-done patch and spends the tokens a second time.

Multi-turn agents need two kinds of fast state with different rules. Session state (cursor, phase, scratch) must survive a node failure, or the agent loses its place. Tool-result caches can be lost, as long as losing them never causes a side-effecting tool to run twice. This guide shows where each belongs on AWS, how to lay out keys for cluster-mode Valkey and Redis OSS, and two patterns tested on a local three-shard cluster: an idempotent cursor and a tool cache with a claim. It merges our earlier ElastiCache scratchpad guide into it.

⚡ TL;DR

• MemoryDB writes are stored in a Multi-AZ transactional log before they’re acknowledged, and primaries stay strongly consistent across failover. Good home for session cursors and scratch.
• ElastiCache for Valkey 9.0+ now offers durability too: synchronous writes (no acknowledged write lost, single-digit ms writes) or asynchronous (up to 10 seconds of writes at risk). Without durability, ElastiCache replication is asynchronous: treat it as a cache.
• DynamoDB stays the source of truth for anything that must be correct under retries: quotas, tool side effects, audit.
• In cluster mode, give every key of a session the same hash tag, e.g. agent:{acme:s1}:meta, or multi-key transactions and scripts fail with CROSSSLOT.
• Advance the cursor with compare-and-set, and claim a tool call with SET NX before running it.

Who this is for: engineers running multi-turn coding agents on Lambda, ECS or Step Functions who keep state between turns. Code is Python with redis-py’s cluster client; it ran against a local three-primary Redis OSS cluster, and the commands used are available on MemoryDB and ElastiCache for Valkey and Redis OSS.

Three stores, three jobs

MemoryDB is compatible with Valkey and Redis OSS. Its docs describe microsecond reads and single-digit millisecond writes, because each write waits for the transactional log. Replicas are eventually consistent, so read session state from the primary. ElastiCache’s durability feature (Valkey 9.0 or later, node-based clusters in cluster mode, Multi-AZ with at least one replica per shard, specific Graviton instance families) brings a similar log to ElastiCache; it isn’t available for ElastiCache Serverless, and it can’t be added to an existing non-durable cluster.

Key layout for cluster mode

MemoryDB always partitions data into shards, and durable ElastiCache requires cluster mode. In cluster mode, a command or transaction that touches several keys only works if they’re in the same hash slot. A hash tag controls that: if a key contains {...}, only the text inside the braces is hashed. So put the tenant and session inside the braces:

"""Hot session state for a multi-turn coding agent on MemoryDB or durable ElastiCache for Valkey.
Both are cluster-mode stores, so every key of one session shares a hash tag and lands in one slot."""
import json
import time

from redis.cluster import RedisCluster
from redis.exceptions import RedisClusterException, ResponseError

SESSION_TTL = 24 * 3600


def k(tenant: str, session: str, *parts: str) -> str:
    # {tenant:session} is the hash tag: only this part is hashed, so MULTI and Lua work across these keys.
    return ":".join(["agent", "{" + f"{tenant}:{session}" + "}", *parts])

Without the tag, the client (or the server) refuses multi-key commands, as the test below shows. Also keep keys out of the model’s hands: build them from your own tenant and session IDs, never from a key name the model supplies, or a prompt injection can read another tenant’s state.

An idempotent session cursor

Agent runtimes retry. A Lambda times out after the write succeeded, Step Functions retries the task, and step 17 arrives twice. The cursor update should only apply if the session is still at the step the caller expects. A short Lua script does the compare-and-advance and writes the scratch in one atomic operation, which works because both keys share the hash tag:

"""Hot session state for a multi-turn coding agent on MemoryDB or durable ElastiCache for Valkey.
Both are cluster-mode stores, so every key of one session shares a hash tag and lands in one slot."""
import json
import time

from redis.cluster import RedisCluster
from redis.exceptions import RedisClusterException, ResponseError

SESSION_TTL = 24 * 3600


def k(tenant: str, session: str, *parts: str) -> str:
    # {tenant:session} is the hash tag: only this part is hashed, so MULTI and Lua work across these keys.
    return ":".join(["agent", "{" + f"{tenant}:{session}" + "}", *parts])


# Advance the cursor only from the step we expect. A retried Lambda or Step Functions task that
# re-sends step 17 after it already committed gets 0 back instead of applying the step twice.
ADVANCE = """
local cur = tonumber(redis.call('HGET', KEYS[1], 'step') or '0')
if cur ~= tonumber(ARGV[1]) then return 0 end
redis.call('HSET', KEYS[1], 'step', ARGV[1] + 1, 'phase', ARGV[2], 'updated', ARGV[3])
redis.call('HSET', KEYS[2], unpack(cjson.decode(ARGV[4])))
redis.call('EXPIRE', KEYS[1], ARGV[5])
redis.call('EXPIRE', KEYS[2], ARGV[5])
return 1
"""


class SessionStore:
    def __init__(self, r: RedisCluster):
        self.r = r
        self.advance_script = r.register_script(ADVANCE)

    def advance(self, tenant, session, expected_step: int, phase: str, scratch: dict) -> bool:
        flat = [x for pair in scratch.items() for x in pair]
        ok = self.advance_script(keys=[k(tenant, session, "meta"), k(tenant, session, "scratch")],
                                 args=[expected_step, phase, int(time.time()), json.dumps(flat), SESSION_TTL])
        return ok == 1

    def load(self, tenant, session) -> dict:
        meta = self.r.hgetall(k(tenant, session, "meta"))
        scratch = self.r.hgetall(k(tenant, session, "scratch"))
        return {"meta": meta, "scratch": scratch}

    def purge(self, tenant, session) -> int:
        keys = list(self.r.scan_iter(match=k(tenant, session, "*"), count=200))
        return self.r.delete(*keys) if keys else 0


def demo_crossslot(r: RedisCluster):
    """Without a shared hash tag, a multi-key command across slots fails in cluster mode."""
    try:
        r.execute_command("MSET", "agent:acme:s1:meta", "x", "agent:acme:s1:scratch", "y")
    except (RedisClusterException, ResponseError) as e:
        return str(e)
    return "no error (keys happened to share a slot)"

On a cache miss for meta, don’t start a fresh session. Load the last archived state from DynamoDB, or stop and ask the user, because silently restarting is how half-done patches get applied twice.

A tool-result cache that doesn’t run tools twice

A common pattern is “check the cache, run the tool, store the result”. Under concurrent retries, two workers both miss the cache and both run the tool. Claim the call first with SET ... NX EX: only one caller gets the claim, the others wait for its result.

"""Tool-result cache with a claim, so concurrent retries don't run the same tool twice.
For tools with real side effects (deploys, payments, PR merges) the source of truth is a
durable ledger such as a DynamoDB conditional write; this cache only saves repeat work."""
import json
import time

from session_store import k

CLAIM_TTL, RESULT_TTL = 120, 900


def run_once(r, tenant, session, idem_key: str, execute, wait_s: float = 30.0):
    key = k(tenant, session, "tool", idem_key)
    if r.set(key, json.dumps({"state": "running"}), nx=True, ex=CLAIM_TTL):
        try:
            result = execute()
        except Exception:
            r.delete(key)                         # let a later retry try again
            raise
        r.set(key, json.dumps({"state": "done", "result": result}), ex=RESULT_TTL)
        return result, "executed"
    deadline = time.monotonic() + wait_s
    while time.monotonic() < deadline:            # someone else holds the claim: wait for their result
        entry = json.loads(r.get(key) or '{"state": "gone"}')
        if entry["state"] == "done":
            return entry["result"], "cached"
        if entry["state"] == "gone":
            return run_once(r, tenant, session, idem_key, execute, wait_s)
        time.sleep(0.2)
    raise TimeoutError(f"tool call {idem_key} still running elsewhere")

This saves duplicate work for read-only or repeatable tools such as tests, searches and builds. It isn’t enough on its own for tools with external side effects: a claim can expire while a slow deploy is still running, and a cache can be lost. Record those in a DynamoDB ledger with a conditional write keyed by the tool-call ID, as in our DynamoDB tool ledger guide, and use the cache only to avoid repeat work.

Test run

import threading

from redis.cluster import RedisCluster

from session_store import SessionStore, demo_crossslot
from tool_cache import run_once

r = RedisCluster(host="127.0.0.1", port=7000, decode_responses=True)   # MemoryDB: ssl=True + ACL user
store = SessionStore(r)
print("without hash tag:", demo_crossslot(r))

print("advance 0->1:", store.advance("acme", "s1", 0, "running_tests", {"last_tool": "rg", "files": "2"}))
print("retry of step 0:", store.advance("acme", "s1", 0, "running_tests", {"last_tool": "rg"}))
print("advance 1->2:", store.advance("acme", "s1", 1, "awaiting_model", {"last_tool": "pytest"}))
print(store.load("acme", "s1"))

calls = []
def slow_tool():
    import time; time.sleep(0.5); calls.append(1); return {"exit": 0, "passed": 41}
out = []
threads = [threading.Thread(target=lambda: out.append(run_once(r, "acme", "s1", "pytest-abc123", slow_tool)))
           for _ in range(3)]
[t.start() for t in threads]; [t.join() for t in threads]
print("tool executions:", len(calls), sorted(o[1] for o in out))
print("purged keys:", store.purge("acme", "s1"))

without hash tag: MSET - all keys must map to the same key slot
advance 0->1: True
retry of step 0: False
advance 1->2: True
{'meta': {'step': '2', 'phase': 'awaiting_model', 'updated': '1791641221'}, 'scratch': {'last_tool': 'pytest', 'files': '2'}}
tool executions: 1 ['cached', 'cached', 'executed']
purged keys: 3
$ redis-cli -c MSET agent:acme:s1:meta x agent:acme:s1:scratch y
CROSSSLOT Keys in request don't hash to the same slot

The retry of step 0 was rejected, three concurrent calls ran the tool once, and purging the session removed its three keys. The CROSSSLOT line is the server’s answer when the same keys have no hash tag.

Connecting from agents: TLS, ACLs and IAM

• Network: both services run in your VPC. Allow the port from the agent runtime’s security group only.
• TLS and users: use TLS in transit and an ACL user with only the commands and key patterns the agent needs (for example ~agent:*).
• IAM authentication: available for Valkey or Redis OSS 7.0 and later (ElastiCache requires Valkey 7.2+ or Redis OSS 7.0+, with TLS). The token is valid for 15 minutes and IAM-authenticated connections are disconnected after 12 hours, so use a client with a credentials provider or re-authenticate long-lived connections.
• Lambda: create the client outside the handler so warm invocations reuse the connection.

Worked example: a refactor that survives failover

1. Turn 1: the agent plans six steps. The orchestrator writes step=0, phase=planning to agent:{acme:s1}:meta.
2. Each step: the tool runs through run_once; side-effecting tools also write a DynamoDB ledger item; then advance(expected_step=n) moves the cursor and records touched files.
3. During step 4, the primary fails over. On MemoryDB (or durable Valkey with synchronous writes), every acknowledged write is still there; the retried step 4 calls advance(4, ...) once and continues.
4. The tool cache for an earlier rg may or may not survive; if not, the search reruns, which is harmless.
5. When the session goes idle, a scheduled job archives meta and scratch to DynamoDB and calls purge.

Troubleshooting

Production checklist

• ☐ Session cursor and scratch in MemoryDB or durable Valkey; caches in ElastiCache; side effects in DynamoDB
• ☐ Hash-tagged keys per session; keys never taken from model output
• ☐ Compare-and-advance on the cursor; claim before running a tool
• ☐ Reads of session state from the primary
• ☐ TLS, least-privilege ACL users or IAM auth with token refresh
• ☐ TTLs on every key; blobs in S3; idle sessions archived and purged
• ☐ Failover drill in staging: kill a primary mid-session and confirm the agent resumes at the same step

FAQ

Isn’t DynamoDB enough for session state?

It can be, and it’s simpler to operate. A Redis-compatible store helps when the agent reads and writes state many times per turn and you want lower latency, or need data structures and scripts.

MemoryDB or durable ElastiCache for Valkey?

Both now offer a Multi-AZ transactional log. Compare engine version, instance types, features you need (ElastiCache durability excludes Serverless, Global Datastores and data tiering) and price for your workload.

Can I use one cluster for both session state and cache?

You can, on a durable cluster, with separate key prefixes and TTLs. Separate clusters let you size, scale and fail over the cache without touching session state.

Why Lua instead of WATCH/MULTI?

Both work in cluster mode with hash-tagged keys. A script does the compare and the writes in one round trip and can’t be interleaved.

Official documentation

• What is Amazon MemoryDB and consistency
• ElastiCache durability and its limitations
• MemoryDB IAM authentication and ElastiCache IAM authentication
• What is Amazon ElastiCache

Related on CheatCoders: DynamoDB transactions for tool ledgers · Evaluating coding-agent changes

Last updated on October 10, 2026