Agent memory is the persisted information an agent can write, retrieve, and reuse across turns within a session, and often across sessions that share an account, workspace, or long-running process. It is distinct from the ephemeral context window, which is the limited working space the model can attend to in a single call. Memory lives outside that window in a store the harness can read from and write back to, and the model treats what it pulls from that store as if it were part of the prompt. Practically, memory may hold user preferences, prior decisions, summaries of long conversations, facts the agent has confirmed, or procedural notes such as 'this repository's tests are run with pytest -q.' Every one of those writes is a design decision: what to capture, when to consolidate, when to correct, who can read and write it, and when to forget it. More stored material is not automatically better memory, because irrelevant or stale entries raise the cost of retrieval and dilute the signal in the context window.[1]
The typical flow looks like this. On each turn, a memory subsystem decides which entries are relevant to the current task and injects them into the prompt. After the turn, a separate step evaluates the exchange and writes new entries or updates old ones, sometimes by summarizing, sometimes by deleting. Three failure modes recur in practice. First, the agent retrieves the wrong slice: a working note from a different project gets pulled into a debugging session, and the model treats it as ground truth. Second, the memory store grows without consolidation, so the most useful prior knowledge is buried under outdated entries. Third, writes happen silently and without review, so a hallucinated 'fact' about an API or a colleague gets locked into the store and replayed later. Mature systems treat memory as code: a tracked artifact with provenance, ownership, and a path for correction rather than a free-form journal.[1][2]
The most durable pattern is to push memory outward, into shared, reviewable artifacts. Skill files, runbooks, and updated documentation carry lessons in a form humans and other agents can audit, version, and correct. Private memory stores still earn their keep for short-lived state such as 'the user prefers metric units in this session,' but anything that should survive the next model upgrade belongs in code-reviewed files. Treat memory as a product surface: a write to memory is a write to a shared system, and every entry should justify its place. The choice between a simple list, a vector store, or a graph-vector hybrid is downstream of the access patterns you actually have, not the pattern you hope to grow into.[1][3]
Agent memory versus context window versus retrieval
The context window is the model's working memory for a single call: fast, finite, and discarded when the call ends. Retrieval, in the RAG sense, fetches documents from an external corpus at query time, often indexed as vectors plus metadata. Agent memory sits in between: it is a writable, persisted store curated by the agent and harness, whose contents are then injected into the context window through a retrieval step. A useful rule of thumb: retrieval answers 'what does the world say,' while memory answers 'what have we already decided or learned.' A RAG system without memory keeps re-discovering the same facts; a memory system without retrieval cannot reach facts the agent has not yet encountered. Confusing the two leads to bloated memory stores that duplicate public documentation, and to retrieval systems that miss the user's standing preferences.
Worked exampleDiagnosing a flaky test that keeps coming back
A coding agent is asked to fix a flaky integration test in a Python repo. Two weeks earlier, a previous session diagnosed the same test, learned the suite must run with pytest -q and that the test depends on a local Postgres fixture, and recorded those findings in memory.
- 01The new session begins. The memory subsystem retrieves the two prior notes and prepends them to the prompt, so the agent never has to rediscover them.
- 02The agent runs pytest -q, sees the fixture failure, and asks the harness to start Postgres before re-running, following the stored procedure.
- 03The fix works. The post-turn consolidation step summarizes the resolution into a new memory entry: 'integration tests need a running Postgres fixture, see runbook/scripts/start_pg.sh.'
- 04A week later, a teammate deletes the runbook script and renames the fixture. The agent retrieves the old note, follows the dead path, and the test fails for an opaque reason. The failure is not a model failure; it is a memory-staleness failure that only a reviewable shared artifact would have caught.
Memory is most useful when it is small, relevant, and auditable. Treat writes as commits, prefer shared reviewable files for anything durable, and never trust a memory note you cannot trace back to a source.