Reports an open weightsA model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.Full definition → model clearing the IMO gold cutoff at about 140 times lower cost than a frontier model. This informs model choice where cost matters for reasoning-heavy work.
Key takeaways · AI-distilled
Cline had proofs graded blind on the 0 to 7 IMO scale by GPT-5.5 and Claude Opus 5, with Gemini 3.1 Pro resolving disagreements; graders saw the problem, rubric, and reference solutions but not the model identity.
In Cline's runs, DeepSeek V4 Flash scored 30/42 for $0.12, while Kimi K3 scored 35/42 for about $5.13 and Claude Fable 5 scored 41/42 for about $17.20; GLM 5.2 scored 21 and Qwen 3.6 35B A3B 16.
Cline took each model's best score per problem across multiple attempts rather than Pass@k, and listed prices cover only the single best run, not the exploration spent fixing tool-calling issues.
Reward-hacking controls included telling models not to use the internet, exposing only a submit-solution tool, giving no prior context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → like Lean artifacts, and checking traces for browser use.
Terms in this piece · Glossary
agent harness — The scaffolding around a model that turns it into a working agent — the loop, the tools it can call, and the rules for when to stop.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
articleDeepseek V4 Flash 0731 Scores 50 On The Artificial Analysis Intelligence Index 10 Points Above Previous Deepseek V4 FlashArtificial Analysis articles
articleDeepSeek-V4 Flash 0731 vs GPT-5.6 Luna on DeepSWE: Cost and CodingTogether AI
articleDeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routingwww.together.ai