Model results
SWE-bench
Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.
Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.
Reviewed recommendation and limits
Use Verified as a historical shared-agent reference. Start with the locked public SWE-bench Pro V2 protocol for contemporary repository issue repair.
Historical Verified with mini-SWE-agent; preserve each imported suite and harness. Imported groups below may use different protocols; this review does not certify each run. OpenAI stopped reporting Verified after finding task defects and memorized solutions. Its 59.4% defect finding concerns 138 selected difficult tasks, not the full 500-task set. Imported Epoch runs have their own protocol.
SWE-bench_Verified · swe bench % resolved
- Source
- Hugging Face eval result
- Version
- Not reported
- Measurement date
- 2026-02-03
- Metric
- % · Higher is better
- Evaluation scope
- Registered model evaluation · Model Card
Positions are withheld because at least one result lacks explicit independent provenance or a documented evaluation scope. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.
Qwen3.5 397B A17B
qwen/qwen3.5-397b-a17b-20260216
76.4%
View modelEvidence Evidence class unreportedQwen3.5-27B
qwen/qwen3.5-27b-20260224
72.4%
View modelEvidence Evidence class unreportedQwen3.5-122B-A10B
qwen/qwen3.5-122b-a10b-20260224
72%
View modelEvidence Evidence class unreportedQwen3.5-35B-A3B
qwen/qwen3.5-35b-a3b-20260224
69.2%
View modelEvidence Evidence class unreported