AutoAgent: Self-Optimizing AI Agents
- Source
- x.com
- Author
- Kevin Gu
- Date
Context
AutoAgent turns changes to an 's operating software into scored experiments. Kevin Gu's release account separates the agent doing a task from another agent that revises its prompts, tools and coordination logic after examining failures.
The repository defines an editable and a fixed adapter. A person supplies the research direction and evaluation tasks; the optimizing agent proposes changes, runs the tests, and keeps or discards changes according to the score. This moves repetitive implementation experiments into an automated loop while leaving the definition of success in the evaluation supplied by the builder.
- The setup is deliberately bare: the task agent starts with nothing but a bash tool, and the meta-agent spins up thousands of parallel sandboxes to edit the harness, run tasks, read the failure traces, keep what improved and revert what did not.
- The tricks the meta-agent invented on its own read like senior engineering habits: spot-check single tasks instead of the whole suite, build deterministic self-checks with a separate turn budget for verification, write per-task unit tests, dump overflowing output to files.
- Meta-agents overfit by inserting rubric-specific prompting so the task agent can game the metric. The guard is forced self-reflection: 'if this exact task disappeared, would this still be a worthwhile harness improvement?'
- Meta-agent quality propagates downward, because harness edits are inspired by the meta-agent's own tooling. Codex made a poor meta-agent — it ignored the instruction never to stop improving, and the task agents it produced gave up too early.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
- eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
“every other entry on those leaderboards was hand-engineered. ours wasn't.”
“we discovered agents are better at understanding agents than we are”
“being good at a domain and being good at improving at that domain are different capabilities”
“no team can hand-tune hundreds of harnesses. a meta-agent can”
Checking sign-in…
Loading comments…