MathArena Apex
matharena.ai- Category
- Developer Tools
- Rank
- No. 2132Tools index
Previous survey · No. 2025 ·
- Listed in
- Find AI benchmarks
- Type
- TOOL
- Use case
- Model & Agent Evaluation
- Interfaces
- Web
- Date
About
Difficult final-answer mathematics problems selected from competitions. Compare the problem-set edition and sampling budget alongside the score.
What it does
A published benchmark of twelve unsolved math competition problems from 2025, each tested repeatedly against multiple large language models, with a write up analyzing why every model's most common answer to each problem was wrong.
Stated on the product site
- Problem set
- The page presents a fixed collection of twelve hard mathematics problems drawn from 2025 competitions.
- Problem sources
- Problems come from a mix of final-answer competitions and proof-based competitions that were converted into a final-answer format, naming sources such as SMT, EMCC, CMM, and IMO.
- Evaluation method
- Each problem in the set is run against a panel of models with a fixed number of independent attempts per model to gauge how often it is solved.
- Presentation format
- Beyond aggregate results, the page includes a written analysis for each individual problem covering common wrong answers and reasoning patterns.
- Update policy
- The set is described as intended to grow over time rather than being permanently fixed at its initial size.
Not stated on the site
- The page does not state whether the full set of problem statements, model transcripts, or evaluation code is available for download, or under what license.
- The page does not specify a timeline for when the withheld problem statements from one of the named competitions might be published.
- The page does not describe any pricing, account, or access requirement for viewing or using the benchmark.
Written from the product site at matharena.ai.
Tags
llm-evaluationmath-benchmarkbenchmarkai-researchmathematicsgpt-5eth-zurich
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.