Amazon MemoryDB: Durable Sub-Millisecond Session State for Multi-Turn Coding Agents

Your coding agent was mid-refactor: session cursor at tool step 17, scratch holding the last rg hits and a staged patch digest — then the ElastiCache primary failed over and every GET session:* returned miss. The planner re-asked the model “what were we doing?”, burned another 40k tokens, and re-applied a half-done patch. Amazon MemoryDB is the unfair advantage: Redis-compatible latency with Multi-AZ transactional durability, so hot session state survives node failure without pretending a volatile cache is your source of truth. Pair with ElastiCache scratchpads for pure caches and DynamoDB tool ledgers for spend/side effects — this post is the durable hot-session layer.

⚡ TL;DR: Put session cursor + tool scratch that must survive failover in MemoryDB. Keep ephemeral tool-result caches in ElastiCache. Keep quota, effect ledger, and cold session archive in DynamoDB. Related: ElastiCache scratchpads, DynamoDB transactions, Step Functions agent graphs.

Three stores, three jobs

Coding agents abuse “Redis” as a kitchen sink. Split responsibilities:

// ✅ Explicit key namespaces per store
const KEYS = {
  // MemoryDB — durable hot session
  sessionMeta: (t: string, s: string) => `mdb:sess:${t}:${s}:meta`,
  scratch: (t: string, s: string, k: string) => `mdb:sess:${t}:${s}:scratch:${k}`,
  // ElastiCache — volatile cache only
  toolCache: (hash: string) => `ec:toolres:${hash}`,
  // DynamoDB — source of truth for money/side effects
  ledgerPk: (t: string, s: string) => `TENANT#${t}#SESSION#${s}`,
};

# ❌ One Redis cluster for everything — failover = lost ledger + lost cache
r.set(f"quota:{tenant}", remaining)          # money in volatile store
r.set(f"effect:{tool_call_id}", "applied")   # side effect without DynamoDB
r.set(f"cache:rg:{q}", json.dumps(hits), ex=60)

Session cursor that survives node failure

A multi-turn agent needs at least:

• cursor / next_tool_ordinal
• active_goal / planner phase
• pointers to last N message IDs (not full blobs if large)
• scratch keys for “files touched”, “last test output hash”

MemoryDB replicates writes with a transactional log across AZs. After primary failure, a replica promotes with data, unlike ElastiCache where you may lose writes that were only in memory.

import { createClient } from "redis"; // works against MemoryDB endpoint

const mdb = createClient({
  url: process.env.MEMORYDB_URL, // rediss://clustercfg....amazonaws.com:6379
  socket: { tls: true },
});
await mdb.connect();

export async function bumpSessionCursor(opts: {
  tenantId: string;
  sessionId: string;
  cursor: string;
  phase: string;
  scratch?: Record<string, string>;
}) {
  const metaKey = `mdb:sess:${opts.tenantId}:${opts.sessionId}:meta`;
  // ✅ MULTI/EXEC for cursor + scratch in one round-trip
  const multi = mdb.multi();
  multi.hSet(metaKey, {
    cursor: opts.cursor,
    phase: opts.phase,
    updated_at: new Date().toISOString(),
  });
  multi.expire(metaKey, 60 * 60 * 24); // 24h — still durable within TTL window
  if (opts.scratch) {
    for (const [k, v] of Object.entries(opts.scratch)) {
      const sk = `mdb:sess:${opts.tenantId}:${opts.sessionId}:scratch:${k}`;
      multi.set(sk, v, { EX: 60 * 60 * 12 });
    }
  }
  await multi.exec();
}

On Lambda restore or Step Functions Task retry, read cursor first before re-issuing tools:

export async function loadSession(tenantId: string, sessionId: string) {
  const meta = await mdb.hGetAll(`mdb:sess:${tenantId}:${sessionId}:meta`);
  if (!meta?.cursor) {
    // Fall back to DynamoDB cold archive — do not invent a fresh cursor
    return loadColdSessionFromDdb(tenantId, sessionId);
  }
  return meta;
}

vs ElastiCache scratchpads vs DynamoDB

Use ElastiCache exactly as documented in the scratchpads post: hash of tool args → cached stdout, short TTL, miss = recompute. Losing that cache is annoying, not incorrect.

Use DynamoDB for anything that must be auditably correct under retries: quota debits, “effect applied”, billing events (TransactWriteItems ledger).

MemoryDB sits in the middle: hot correctness — if the node dies mid-turn, the next Invoke should resume at the same cursor without re-prompting the model from zero.

IAM, TLS, and agent sandbox wiring

MemoryDB wants TLS and typically VPC-only access. Tool Lambdas (SnapStart or container) and CodeBuild sandboxes need:

1. Security group ingress from agent runtime SG
2. IAM is not Redis AUTH replacement — still use AUTH tokens / ACLs
3. Per-tenant key prefixes enforced in application code (Redis does not know your tenants)

# ✅ boto3 create cluster sketch + app-level tenant isolation
import boto3

memorydb = boto3.client("memorydb")

# Ops once; agents never CreateCluster
# memorydb.create_cluster(... ACLName="agents-acl", TLSEnabled=True)

import redis

r = redis.Redis(
    host=os.environ["MEMORYDB_HOST"],
    port=6379,
    ssl=True,
    username="agent-app",
    password=os.environ["MEMORYDB_AUTH"],
    decode_responses=True,
)

def scratch_set(tenant: str, session: str, key: str, value: str, ttl: int = 3600):
    # ✅ Never allow tenant to pass raw Redis keys from the model
    safe = key.replace(":", "_")[:64]
    r.setex(f"mdb:sess:{tenant}:{session}:scratch:{safe}", ttl, value)

// ❌ Model-supplied Redis key — prompt injection → cross-tenant read
await mdb.get(event.keyFromModel);

Authorize tools with Verified Permissions before any MemoryDB write that mutates session phase.

Production patterns that age well

1. Write-through cursor: after every successful tool ledger commit, bump MemoryDB cursor in the same application transaction boundary (best-effort then reconcile from DynamoDB on mismatch).
2. Periodic cold archive: Lambda on schedule dumps mdb:sess:*:meta to DynamoDB/S3 for sessions idle > N hours.
3. Size caps: store hashes and S3 keys in scratch, not 2MB test logs — put blobs in S3, pointers in MemoryDB.
4. Failover drills: kill the primary in staging and prove the agent resumes without a full replan.
5. Cost: MemoryDB is pricier than ElastiCache — only promote keys that need durability; keep bulk caches on ElastiCache.

// ✅ After DynamoDB ledger success, advance durable cursor
await commitToolLedger({ ... }); // source of truth
await bumpSessionCursor({
  tenantId,
  sessionId,
  cursor: nextCursor,
  phase: "awaiting_model",
  scratch: { last_tool: toolName, last_digest: digest },
});

Checklist

• [ ] Split ElastiCache (volatile cache) vs MemoryDB (durable hot session) vs DynamoDB (ledger/SoT)
• [ ] Session cursor + mid-turn scratch live in MemoryDB with TLS + ACL
• [ ] Never accept raw Redis keys from the model; prefix by tenant/session
• [ ] On miss, fall back to DynamoDB cold archive — do not reset the agent blindly
• [ ] Drill primary failover; confirm agent resumes at same cursor
• [ ] Cap value sizes; put large tool outputs in S3 with MemoryDB pointers