Open Dataset OpenThoughts-Agent-v2 Generalizes Across 7 Agent Benchmarks
- Source
- x.com
- Date
Most open agentic datasets target one benchmark. In compute-controlled comparisons, OpenThoughts-Agent-v2 leads at every training set size, and generalizes across seven agentic benchmarks. Check it out! https://t.co/BF0Fv6Ug51
Context
Most open agentic training datasets are built around one 's task format, which the OpenThoughts-Agent paper argues is why models trained on them often fail to transfer to similar tasks outside that format. Stanford SAIL's post points to OpenThoughts-Agent-v2, built from a systematic set of ablations across the data pipeline's stages rather than tuned to one target task.
The authors report that Qwen3-32B on their resulting 100,000-example dataset reaches 44.8% average accuracy across seven agentic benchmarks, including , Terminal-Bench 2.0, and GAIA-127, a 3.9 percentage point improvement over the strongest prior open dataset's model in a compute-controlled comparison. These are the authors' own reported comparisons under their stated compute controls.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
- SWE-bench — The standard benchmark for AI coding agents: real GitHub issues from real repositories, scored by whether the agent's patch passes the project's own tests.
Checking sign-in…
Loading comments…