Model results

SWE-bench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Reviewed recommendation and limits

Use Verified as a historical shared-agent reference. Start with the locked public SWE-bench Pro V2 protocol for contemporary repository issue repair.

Historical Verified with mini-SWE-agent; preserve each imported suite and harness. Imported groups below may use different protocols; this review does not certify each run. OpenAI stopped reporting Verified after finding task defects and memorized solutions. Its 59.4% defect finding concerns 138 selected difficult tasks, not the full 500-task set. Imported Epoch runs have their own protocol.

Publisher results OpenAI audit of SWE-bench Verified

← All evaluation groups

SWE-bench Verified

Source: Epoch AI Benchmarking Hub
Version: Not reported
Measurement date: Not reported
Metric: % · Higher is better
Evaluation scope: Epoch AI run · task v2 · effort high · result date not supplied

Exact score ties share a position. Positions do not establish statistical significance. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

#1Gemini 3.5 Flash
google/gemini-3.5-flash-20260519
79.3%
View modelEvidence independent
#2GPT-5.4
openai/gpt-5.4-20260305
76.9%
View modelEvidence independent
#3GPT-5.3-Codex
openai/gpt-5.3-codex-20260224
74.8%
View modelEvidence independent
#4GPT-5.2
openai/gpt-5.2-20251211
73.8%
View modelEvidence independent
#5GPT-5
openai/gpt-5-2025-08-07
73.6%
View modelEvidence independent
#6GPT-5.1
openai/gpt-5.1-20251113
68%
View modelEvidence independent