Model results
ArXivMath
Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.
Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.
Evaluation groups
Open a group to see its exact protocol and scores. Different configurations remain separate.
No model rankings are available in this snapshot. Reported results remain available below.
Other reported results
These reports have no model ranking. Open a group for its evidence and the reason positions are withheld.
- Limite 1B - Violettounversioned
ArXivMath May 2026 · Paradigma model card (vendor-reported)
Single-model result · Positions withheld
Measurement date 2026-09-21
Scope · Vendor-reported; evaluated by Paradigma; percentages, higher is better; evaluation settings not stat…
Vendor-reported; evaluated by Paradigma; percentages, higher is better; evaluation settings not stated (the card recommends temperature 0.6, top_p 0.95 for use); read from the model card's evaluation table image; result date not stated by the publisher, dated to the model-card publication
- Claude Fable 5.12026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Claude C…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Claude Code 2.1.267; configuration=Claude-Fable-5.1 (low); reasoning effort low; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 10.58%; cost per problem $9.15; model configuration https://matharena.ai/models/anthropic_fable_51_low; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint anthropic/claude-fable-5.1; publisher does not expose a dated provider revision
- Claude Fable 5.12026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Claude C…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Claude Code; configuration=Claude-Fable-5.1 (high); Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 6.03%; cost per problem $12.77; result date not supplied; captured 2026-09-17; publisher flags model release after competition release
- Claude Opus 5.52026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Claude C…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Claude Code 2.1.280; configuration=Claude-Opus-5.5 (high); reasoning effort high; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 9.02%; cost per problem $1.68; model configuration https://matharena.ai/models/anthropic_opus_55_high; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint anthropic/claude-opus-5.5; publisher does not expose a dated provider revision
- DeepSeek V4.1 Flash2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=MathAren…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=MathArena "deepcode" harness 0.3.1; configuration=DeepSeek-V4.1-Flash (Max); reasoning effort max; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 9.11%; cost per problem $0.43; model configuration https://matharena.ai/models/deepseek_deepseek_v41_flash; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint deepseek/deepseek-v4.1-flash; publisher does not expose a dated provider revision
- Gemini 3.8 Flash2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Antigrav…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Antigravity CLI; configuration=Gemini 3.8 Flash; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 9.05%; cost per problem $0.92; result date not supplied; captured 2026-09-17; publisher flags model release after competition release
- GPT-6 Astra2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex 0.…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex 0.153.3; configuration=GPT-6 Astra (low); reasoning effort low; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 6.54%; cost per problem $1.22; model configuration https://matharena.ai/models/openai_gpt_6_astra_low; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint openai/gpt-6-astra; publisher does not expose a dated provider revision
- GPT-6 Astra2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex; c…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex; configuration=GPT-6 Astra (max); Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 5.83%; cost per problem $3.63; result date not supplied; captured 2026-09-17; publisher flags model release after competition release
- GPT-6 Sol2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex 0.…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex 0.153.3; configuration=GPT-6 Sol (max); reasoning effort max; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 6.63%; cost per problem $4.82; model configuration https://matharena.ai/models/openai_gpt_6_sol; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint openai/gpt-6-sol; publisher does not expose a dated provider revision
- GPT-6.1 Sol2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex 0.…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=Codex 0.159.0; configuration=GPT-6.1 Sol (max); reasoning effort max; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 4.78%; cost per problem $0.93; model configuration https://matharena.ai/models/openai_gpt_61_sol; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint openai/gpt-6.1-sol; publisher does not expose a dated provider revision
- Grok 4.72026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=MathAren…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=MathArena "grok" harness 1.0.40; configuration=Grok 4.7 (xhigh); reasoning effort xhigh; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 9.78%; cost per problem $7.95; model configuration https://matharena.ai/models/xai_grok_47; result date not supplied; captured 2026-10-15; publisher flags model release after competition release; publisher model endpoint x-ai/grok-4.7; publisher does not expose a dated provider revision
- Kimi K32026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=publishe…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=publisher-selected native harness (exact harness version unspecified); configuration=Kimi K3 (Think); Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 12.64%; cost per problem $5.77; result date not supplied; captured 2026-09-17
- Muse Spark 1.32026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=publishe…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=publisher-selected native harness (exact harness version unspecified); configuration=Muse Spark 1.3; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 9.91%; cost per problem $1.15; result date not supplied; captured 2026-09-17; publisher flags model release after competition release
- Qwen3.8 Max2026-08-native-harness
ArXivMath August 2026 · MathArena (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=publishe…
Final-answer correctness; 57 questions; LLM equivalence judge; August 2026 dataset; harness=publisher-selected native harness (exact harness version unspecified); configuration=Qwen3.8-Max; Python and SageMath tools; no internet; budget/time-limit instructions varied across models, see methodology https://matharena.ai/arxiv_august/; 95% CI 12.82%; cost per problem $5.76; result date not supplied; captured 2026-09-17; publisher flags model release after competition release