MIT lets 13 coding agents explore an island; their memory notes carry skills
- Source
- arxiv.org
- Author
- MIT_CSAIL
- Date

Context
Researchers at MIT, in a paper shared by MIT CSAIL, report that AI coding agents left with no task in a simulated world wrote down techniques they discovered, and those notes later made them better at assigned goals. Their , called Eko, takes an unmodified coding assistant (in the main experiment, Anthropic's Claude Code running Claude Opus 4.7) and adds three things: an orchestrator that keeps telling it to continue, a text interface for moving a body around an island, and a workspace of memory files. Thirteen agents each ran for 30 hours. Every hour the assistant's and the island were reset, so only the notes carried experience forward. According to the paper, the agents climbed hills, stacked blocks into towers, drew patterns, invented sports and ran experiments on the world's physics.
The authors then tested whether those notes amounted to learning. They gave each 30-hour agent, and 13 fresh agents whose memory files held only the starting templates, one hour on each of four goals. Both groups used the same with no weight changes, so any difference had to come from the files. The experienced agents generally did better. Asked to build as many five-block towers as possible, 8 of 13 built more than two, against 2 of 13 fresh agents, and they placed an average of 10.3 objects on a tall landmark called the Spire, against 3.5. The island starts with only enough blocks for two towers. More come from throwing a rock hard enough, a technique almost all 13 agents found while exploring. When the researchers erased every mention of it from one agent's memory files, that agent's tower count fell from six to two on average, while its scores on the other three goals did not change.
The same notes can also hold an agent back. On the tallest-tower goal the two groups scored about the same, and two experienced agents topped out at four blocks, below the shortest tower of any fresh agent. The authors attribute such cases to false beliefs written into memory: one agent's notes said that placing blocks could not stack them reliably, and erasing that belief significantly raised its tallest-tower score. Another agent never recorded how to jump onto raised platforms and never reached one, during exploration or . The behavior also depends on the model, since five agents on OpenAI's Codex with GPT-5.5 and five on Kimi Code took up a much narrower range of activities than the Claude Code agents. The researchers say they designed the world with these skills in mind, so the paper shows written memory carrying discovered techniques in that one setting rather than across tasks in general.
- Setup — The combination of a model, a harness, its connections and the path the task's data takes, considered together for a task.
- Session — One conversation or piece of work as a harness keeps it: a record of messages, tool calls and observations that can be saved and resumed.
- Model — A computational system that uses learned patterns to produce an output from information supplied to it.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
videoYour AI Agent Keeps Making the Same Mistakes - Sparsh Jain, CentralAgent AIAI Engineer
article(Im)Paired Programming: Coding Agents Improve Productivity but Harm UnderstandingNishant Balepur, Connor Baumler, Valerie Chen, Eunsol Choi, Rachel Rudinger, Jordan Lee Boyd-Graber
videoTurn 10,994 Notes Into Memory - Paul Iusztin, Decoding AI & Louis-François Bouchard, Towards AIAI Engineer
Checking sign-in…
Loading comments…