Shows a frontier model finding and exploiting a degenerate strategy to max a benchmark, and that multi-agentUsing several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.Full definition → scaffolds did little for learning. Both matter when interpreting AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition →evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition → scores.
Key takeaways · AI-distilled
All of GPT-6 Astra's top scores used one card that bypasses the game's default time-constraint expectations. Epoch found no direct evidence of memorization or cheating, and its top human baseline player used the same card to reach a maximum score.
Epoch banned the card in the default setting because, used fully, it solves the whole game and skips the tactical play the benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → is meant to measure. Its second-ranked human player scored 100% without it, so a maximum score stays reachable.
Epoch concedes EBR-bench as launched had a single clear solution, undercutting its aim to be a better proxy for scientific discovery than puzzle-like benchmarks with one answer.
Terms in this piece · Glossary
multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
