An independent evaluation of Gemini 2.5 Deep Think shows gains in background knowledge and precise computation, and usefulness as a research assistant. It also finds limited creativity on harder proofs and citation errors, which helps set expectations before relying on it.
Key takeaways · AI-distilled
In Epoch's October 2025 evalA repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.Full definition →, Gemini 2.5 Deep Think set FrontierMath records of 29% on Tiers 1-3 and 10% on Tier 4, short-answer problems that need deep background knowledge and precise computation.
The publicly available Deep Think, a version of the model that scored gold-medal-equivalent at the IMO, scores bronze-medal-equivalent on the 2025 IMO, and Epoch did not see it solve IMO problems needing more creativity or intricate proofs.
On geometry, Deep Think relies much less on coordinate systems than other models, which Epoch reads as a more conceptual approach.
Across the tests Deep Think gave multiple incorrect citations to the mathematical literature, including sources that did not exist, and in one case got a grade-school word problem wrong with an error resembling classic human cognitive biases.
Google commissioned the evaluation; Epoch says it kept editorial control and calls it the most comprehensive third-party evaluation of a high-compute model setting to its knowledge.
Terms in this piece · Glossary
eval — A repeatable test for AI quality — a set of tasks plus scoring — used the way software teams use test suites, because model output is too variable to judge by eyeballing.
