Total Recall: Agent Memory and Harness Engineering — Ignacio Martinez, Oracle
Source
AI Engineer
Author
AI Engineer
Date
Key takeaways · AI-distilled
Ignacio Martinez (Oracle) frames a agent harnessThe scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.Full definition → as a model wrapped in seven engineering layers: the model itself is 'the frozen part of the reasoning' (you rent the weights, you don't control them), and harness engineering turns that nondeterministic core into repeatable output.
On agent memory storage, files suit a model's instincts and are cheap to append to but lack transactional consistency; for coordinating parallel agents so they don't clobber each other's work, the room's answer reinvents what databases solved: work trees.
The talk borrows 'Umwelt' (Jakob von Uexkull's term for an organism perceiving reality only through its own senses) to describe an agent's context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →: its effective 'lens' on the world is exactly whatever institutional knowledge someone remembered to write down.
A 'hysteresis variable' is proposed as a harness parameter: it encodes how much patience the system extends to a model before deciding to give up on a task, rather than retrying or escalating indefinitely.
'agent skillA reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.Full definition → promotion' is described as compressing a workflow that once took four hours of manual work into a faster, reusable version, then retiring the original slow path once the distillationTraining a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.Full definition → one is proven.
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
agent skill — A reusable instruction file that teaches an agent how to do one job well — the procedure, the tools, and what counts as done.
distillation — Training a small, cheap model to imitate a big one's outputs, keeping much of the capability at a fraction of the cost.
Why it matters
This talk gives a concrete seven-layer model for building agent harnesses, including why file-based memory needs git worktrees for parallel agents and how to structure a semantic memory layer, practical architecture guidance beyond a bigger context window.
videoHarness Engineering: Building the Production Cage for Powerful Domain Agents — Mike Chambers, AWSAI Engineer
videoNo Memory, No Harness: Why the Database Is the Last Line of Defense — Kay Malcolm, OracleAI Engineer
videoEvolution of agentic surfaces — Gagan Bhat & Isabella Kai He, AnthropicAI Engineer