From Context to Memory: Your Agents Need a Real Memory Layer — Anders Swanson, Oracle
Source
youtube.com
Author
AI Engineer
Date
Why it matters
Lays out the memory architecture (formation, recall, evolution, governance) and failure modes, so you can judge whether scratch files and grep are enough or you need a real memory store.
Key takeaways · AI-distilled
Swanson argues context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → windows are not memory and bigger windows don't fix that; markdown scratch pads and grep don't scale, and teams that keep pushing them end up building their own database.
He splits AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → memory into four types (episodic, semantic, procedural and working state) stored in a schema with vectors, text indexes, graph links, time and audit history.
Memory formation should apply redaction before anything is written, and recall should use hybrid searchCombining keyword and semantic search in one ranked result, so exact terms and paraphrases both find what they should.Full definition → that blends several retrieval methods.
He names stale, mis-scoped, poisoned and conflicting memories as the main failure modes, and says feedback is what closes the loop so memories can evolve.
Terms in this piece · Glossary
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
hybrid search — Combining keyword and semantic search in one ranked result, so exact terms and paraphrases both find what they should.
videoWhy Your Company Needs a Context Graph (and How to Build It) — Gil Feig, Merge
videoSmall Models, Big Results: Training a Finance Agent for Under $500 — Charles Dickens, Snorkel AI
videoGeneration Is Cheap, Review Is Expensive: How to Stop Shipping AI Slop — Gabriel Martinez, G2i
videoWhy 80% Reliability Isn't Good Enough — Felipe Blanes, Amazon AGI Lab
