Day 13: Memory: Scratchpad, Session, and Long-Term Stores

“Give the agent memory” is not a feature until you say which memory. Day 13 separates scratchpad (this turn’s working notes), session (conversation state), and long-term stores (user/project preferences, durable facts) — and assigns each to the prompt, Redis/DynamoDB, or a vector/DB hybrid on purpose.

⚡ TL;DR: Scratchpad = ephemeral in-state. Session = keyed by conversation ID with TTL. Long-term = explicit writes with provenance and deletion. Do not retrieve long-term memories into the prompt without ACL checks and size budgets (Day 1).

Three tiers

# ✅ Explicit writes — models should not "silently remember"
def remember(user_id: str, key: str, value: str, source: str):
    db.put({
        "pk": user_id,
        "sk": f"mem#{key}",
        "value": value,
        "source": source,  # "user_explicit" | "approved_summary"
        "created_at": now(),
    })

❌ Automatic embedding of every chat turn into a forever vector store with no retention policy.

What belongs where

• Scratchpad: current hypothesis, files touched, failing test name.
• Session: open ticket ID, repo checkout SHA, approval state for HITL.
• Long-term: “prefer pnpm”, “never touch infra/prod”, team coding standards — with user visibility and delete.

Session state should include tool idempotency namespaces and pending async job IDs (Days 11, 16, 17).

Retrieval into the prompt

Long-term memory is RAG with higher stakes. Apply the same cite-or-show discipline: show the user which memories were injected, budget tokens, and allow purge. For multi-tenant apps, memory keys must be tenant-scoped (Day 23).

On AWS, a common pattern is DynamoDB for session + preferences, OpenSearch/pgvector for semantic memory of approved docs only — not raw chat sludge.

Closing checklist

• [ ] Name the three tiers in your architecture doc
• [ ] TTL on session state; retention policy on long-term
• [ ] Explicit remember/forget tools or UI
• [ ] ACL before memory retrieval
• [ ] Token budget for injected memories
• [ ] No silent forever embedding of raw transcripts

Worked example: PR-bot session record

{
  "session_id": "pr-8842",
  "repo": "acme/api",
  "head_sha": "abc123",
  "scratch": {"failing_test": "auth.spec.ts"},
  "pending_job": null,
  "memories_used": ["pref:package_manager=pnpm"]
}

When the head SHA changes, invalidate scratch derived from the old diff. Memory of package manager may persist.

Failure modes to watch

• Session as infinite prompt transcript.
• Cross-user memory bleed from missing keys.
• Unreviewed auto-memories that teach bad habits.
• No forget path for GDPR/internal deletion requests.

Field notes from production

Session records should include model ID and policy version so replay/debug works. When you change memory schemas, migrate with version fields (mem_schema=2). Never share scratchpads across users even inside the same tenant admin view.

Implementation sketch

# Implementation sketch: tiered read
def load_context(session_id, user_id):
    sess = dynamo.get_session(session_id)
    mems = dynamo.query_mem(user_id, limit=5)
    return pack(scratch=sess.scratch, session=sess.summary, memories=mems)

Operator addendum

Summaries used as session memory should carry a hash of source turns. If the user disputes a memory, you can show the summary and regenerate from sources instead of arguing with a ghost.

Memory UX for trust

Show users a “what I remember” panel with delete buttons. Silent long-term memory feels creepy and goes wrong quietly. For team-shared memories (coding standards), require maintainer role to write. Separate personal preferences from org policy in the data model so a user cannot forget a mandatory security memory. When injecting memories into the prompt, prefix with a short header listing keys used — mirrors citation discipline from RAG.

Extended discussion

Return to the core angle for Day 13: What belongs in the prompt vs DynamoDB vs a vector store. That sentence is the acceptance lens for every design review this week. If a proposed change does not make this angle easier to measure or enforce, it is a distraction.

Write down three metrics you will look at after shipping Day 13 ideas, schedule a 45-minute readout, and archive the notes next to the eval artifacts. Architecture without a readout becomes slideshow archaeology.

Pair this day with the adjacent lessons in the series navigation below. Forward links exist so you can keep momentum; backward links exist so you can repair foundations when a later lab fails for boring earlier reasons.

Practically, allocate half a day to implement the smallest vertical slice, half a day to wire measurement, and refuse to polish UI until both are done. This ordering is how bootcamp projects stay honest under time pressure.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 13 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 13 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 13 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Revisit assumptions whenever the model ID, embedding ID, or index alias changes — treat those as breaking changes for Day 13 behaviors, with the same seriousness as a database migration. Canary first, then promote.

Series navigation

← Day 12 · Day 14 →

Last updated September 11, 2026