Concrete evidence that coding agents can reimplement large codebases from a checkable spec, with gains from more inference compute, which shapes what long-horizon tasks you can delegate.
Key takeaways · AI-distilled
In MirrorCode's preliminary results, Claude Opus 4.6 autonomously reimplemented gotree, a bioinformatics toolkit of about 16,000 lines of Go with 40+ commands, a task the authors guess would take an unassisted engineer 2-17 weeks.
Agents get execute-only access to the original CLI program and visible tests but no source code, and end-to-end tests check for exactly matching outputs, with no human steering during runs.
The authors argue common SWE benchmarks measure short tasks: only about 100 of SWE-benchThe standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.Full definition → Pro's 731 tasks involve diffs over 100 lines.
The benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition →, co-developed with METR, spans 20+ target programs from Unix utilities to interpreters and compression; results so far show continued gains from inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → scaling on larger projects.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
