Model results

SWE-bench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Reviewed recommendation and limits

Use Verified as a historical shared-agent reference. Start with the locked public SWE-bench Pro V2 protocol for contemporary repository issue repair.

Historical Verified with mini-SWE-agent; preserve each imported suite and harness. Imported groups below may use different protocols; this review does not certify each run. OpenAI stopped reporting Verified after finding task defects and memorized solutions. Its 59.4% defect finding concerns 138 selected difficult tasks, not the full 500-task set. Imported Epoch runs have their own protocol.

Publisher results OpenAI audit of SWE-bench Verified

← All evaluation groups

SWE-bench Verified

Source: Epoch AI Benchmarking Hub
Version: Not reported
Measurement date: Not reported
Metric: % · Higher is better
Evaluation scope: Epoch AI run · task v2 · result date not supplied

Exact score ties share a position. Positions do not establish statistical significance. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

#1Claude Opus 4.6
anthropic/claude-opus-4-6
78.7%
View modelEvidence independent
#2Qwen3.7 Max
qwen/qwen3.7-max-20260520
77.3%
View modelEvidence independent
#3Claude Opus 4.5
anthropic/claude-opus-4-5
76.7%
View modelEvidence independent
#3Kimi K2.6
moonshotai/kimi-k2.6-20260420
76.7%
View modelEvidence independent
#3Qwen3.6 Max Preview
qwen/qwen3.6-max-preview-20260420
76.7%
View modelEvidence independent
#6Gemini 3 Flash Preview
google/gemini-3-flash-preview-20251217
75.4%
View modelEvidence independent
#7Claude Sonnet 4.6
anthropic/claude-sonnet-4-6
75.2%
View modelEvidence independent
#8GLM 5.1
z-ai/glm-5.1-20260406
74.2%
View modelEvidence independent
#9Kimi K2.5
moonshotai/kimi-k2.5-0127
73.8%
View modelEvidence independent
#10Claude Opus 4.1
anthropic/claude-opus-4-1
73.3%
View modelEvidence independent
#11GLM 5
z-ai/glm-5-20260211
72.1%
View modelEvidence independent
#12Claude Sonnet 4.5
anthropic/claude-sonnet-4-5
71.3%
View modelEvidence independent
#13Claude Opus 4
anthropic/claude-opus-4
70.7%
View modelEvidence independent
#14Qwen3.6 Plus
qwen/qwen3.6-plus-04-02
57.9%
View modelEvidence independent
#15Gemini 2.5 Pro
google/gemini-2.5-pro
57.6%
View modelEvidence independent
#16GPT-4.1
openai/gpt-4.1-2025-04-14
48.5%
View modelEvidence independent
#17GPT-4o (2024-11-20)
openai/gpt-4o-2024-11-20
31%
View modelEvidence independent