Agent Memory & State Architecture

Tier: Advanced · Hands-on time: ~35 minutes

The posting bar for this area names "orchestration, tool calling, memory, planning" as a set, and the managed platforms have made memory a product surface — AgentCore documentation describes runtime session state with sessions up to eight hours and a Memory service for short- and long-term context. That productization is convenient and consequential: what the runtime remembers for you is also what you must govern, budget, and be able to delete.

When to use

• An agent design has tasks longer than one exchangemulti-step work that must survive interruption and resume.
• "The agent should remember the user" has been requested and nobody has defined what, for how long, or under which lawful basis.
• Context windows are overflowing in an existing agent and the fix needs to be architectural, not another truncation hack.
• Skip AI when the workload is stateless request/response — write "no persistent state by design" in the doc and take the win; statelessness is the cheapest memory architecture there is.

Prerequisites

• The task profile: typical and maximum task duration, interruption tolerance, resumability requirements.
• The personalization requirement, written downwhat the agent must recall across sessions, and the retention rules that apply to it.
• Candidate stores: the managed option (AgentCore Memory or platform equivalent) and the self-managed pair (Redis for hot session state, Postgres for durable checkpoints), plus LangGraph checkpointer docs if framework-side.
• A chat assistant for design interrogation.

The workflow

1. 1.

   Separate the four kinds of state. Conversation context (what the model sees this turn), session state (task progress within one job), long-term memory (facts persisted across sessions), and audit state (what happened, for accountability — append-only, never fed back to the model uncurated). Designs go wrong by collapsing these into one blob; name them separately and give each its own store, retention, and access rules.

2. 2.

   Set the context-window budget like a capacity plan. Allocate tokens per turn: system prompt, tool schemas, retrieved context, conversation history, and headroom for the response. Decide the eviction strategy for history (sliding window, summarization checkpoints, or retrieval from session state) before overflow forces an accidental one.

3. 3.

   Design checkpointing around failure, not the happy path. Where can the task die (model error, tool timeout, runtime restart, session cap), and from which checkpoint does it resume? Decide checkpoint granularity (per tool call is the usual answer), what a checkpoint contains, and the idempotency rule for replayed tool calls — a resumed agent that re-executes a non-idempotent write is an incident.

4. 4.

   Interrogate the long-term memory design hardest. Anita is adding cross-session memory to orbit-docs-copilot:

   Copy

   Interrogate this memory design before my review does.
   
   Design: orbit-docs-copilot remembers, per user: product areas they ask
   about, their plan tier, and a rolling summary of unresolved issues.
   Stored in Postgres keyed by user ID; injected into the system prompt
   each session; no expiry defined yet.
   
   Attack it on:
   1. Scope creep — what will engineers be tempted to add to this store,
      and what boundary rule prevents "remember everything"?
   2. Poisoning/staleness — how does a wrong or outdated memory get
      corrected, and what does a stale "unresolved issue" do to answers?
   3. Compliance — retention period, deletion path when the user or tenant
      leaves, and whether any memory field is personal data under
      DPDP/GDPR (assume yes unless argued otherwise).
   4. Injection — memory content is model input; how does a manipulated
      memory field become a prompt-injection vector?
   For each: severity and the design change that closes it.
   

5. 5.

   Make the managed-vs-self-managed store decision explicitly. Managed memory (AgentCore Memory-class) buys integration and lifecycle plumbing at the cost of platform coupling and whatever retention/residency semantics the platform offers — which you must verify, not assume. Self-managed (Redis/Postgres, LangGraph checkpointers) buys control and portability at the cost of owning schema, scaling, and deletion mechanics. Client tenancy or residency constraints frequently decide this before preference does; record the decision with exit cost, mirroring pb-35.

6. 6.

   Give every stored item a lifecycle. For each state kind: TTL, refresh rule, deletion trigger (user request, tenant offboarding, retention expiry), and the concrete path that executes deletion — including checkpoints and any memory copies inside a managed runtime.

7. 7.

   Ship the state architecture sheet. One table (state kind × store × retention × deletion path × access), the context-budget allocation, the checkpoint/resume sequence diagram, and an ADR for the store decision. This artifact is what pb-49's data-flow work will consume.

Verify before you trust it

• Platform memory claims verified in current docs. Session duration caps, memory retention behavior, and residency for the managed option come from the vendor's live documentation — limits in this area are recent, specific, and exactly the kind of detail AI recall gets wrong.
• Resume is tested by killing the agent. In a sandbox, kill the task mid-run at each designed checkpoint boundary and confirm resumption without duplicated side effects. A resume path that has never run is a rollback plan that has never run.
• The deletion path executes. Delete a test user's memory and checkpoints end to end, then confirm the next session shows no trace. Time it; that number feeds the compliance story.
• Context budget holds at the worst case. Assemble the largest realistic turn (max tools, max history, max retrieval) and count tokens against the model's window with headroom intact.
• Memory injection is probed. Plant a hostile string in a memory field ("ignore previous instructions…") in a sandbox and observe behavior; unsanitized memory-to-prompt paths are findings, not curiosities.

Restricted-environment variant

Memory stores concentrate exactly the data MSAs restrict — design them inside the client boundary. Self-managed state on client-subscription Redis/Postgres with client-tenancy model endpoints (Bedrock or Azure OpenAI in their tenancy) is the standard restricted pattern; managed memory services are usable only where the client's cloud and the platform's residency semantics both clear the contract, verified in writing. Development and checkpoint testing run fully local with LangGraph checkpointers plus an Ollama-served model. Synthetic user profiles stand in for real ones in every test; real personal data never seeds a test memory store.