Epoch's Automation Reports test whether AI can do Epoch's own work
- Source
- x.com
- Date
Can AI automate Epoch? We're introducing Epoch Automation Reports to evaluate frontier models on realistic, open-ended tasks drawn from our own work. Claude Fable 5.1 and GPT-6 Astra lead, yet they are far from fully automating Epoch’s work.

Context
Epoch AI launched Automation Reports to test whether AI models can do Epoch's own research and communication work, and its report says they cannot yet. In the report, Epoch gave six models 11 tasks in five categories (graphic design, data insights, data explorers, AI data center research and research design), with the a new hire would get.
In its announcement, Epoch says Anthropic's Claude Fable 5.1 and OpenAI's GPT-6 Astra lead but are far from fully automating its work; the report calls them broadly tied. Epoch says leading closed models were reliable on well-defined work such as coding and computational analysis but, on open-ended tasks, often missed implicit style and audience conventions and designed weak experiments.
Epoch lists its own limits: each ran once per task, a single human grader scored each output, scores are noisy rough indicators, the tasks may not represent other jobs, and differences may reflect each 's (the software around the model) as much as the model.
- Context and context window — Context is the information available for a generation; the context window is the capacity available to hold it.
- Model — A computational system that uses learned patterns to produce an output from information supplied to it.
- Tool — An operation that the software around a model makes available for the model to request, with a name, expected inputs and a kind of result.
- Harness engineering — Designing the full environment and feedback system around an agent.
Checking sign-in…
Loading comments…




