Offers a task-level framework for tracking AI R&D automation, instead of relying on compute and benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition → proxies.
Key takeaways · AI-distilled
The authors argue common AI timeline proxies, effective-compute extrapolation and METR-style task time horizons, miss much of real research work, such as coordinating several projects across a million lines of code without clear success criteria.
O*NET, which economists use to describe about 1,000 US jobs, is too coarse for this: its closest job's top task is to 'analyze problems to develop solutions involving computer hardware and software', which barely distinguishes anything an AI engineer does.
Epoch's first-draft taxonomy splits a frontier lab's research workflow into six categories and over sixty tasks; for example, the 'Run' category covers monitoring runs, hardware infrastructure operations and inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → reliability engineering.
Each task carries a subjective 0-5 rating of how much current AI automates it, which the authors hope could become a trackable metric such as the fraction of tasks automated or per-task researcher uplift, possibly measured via surveys or usage logs.
The authors flag limits: ratings are subjective, needed tasks may change if AIs stop needing human-style steps like writing memos, and future AI research may not resemble current human workflows at all. They rejected organizing by pretrainingThe first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.Full definition →/post-training stages.
Terms in this piece · Glossary
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
