Model results
DeepSWE
Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.
Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.
Reviewed recommendation and limits
Use as supplementary evidence for original repository engineering tasks, with the grading audit beside the result.
DeepSWE v1.1. Grade committed patches in a separate verifier container. Preserve agent, effort, repetitions and completed-trial denominator; keep v1 and later revisions separate.. Imported groups below may use different protocols; this review does not certify each run. Samaritan Research reports false-negative grading defects in at least 23 of 113 v1.1 tasks. A subsequent repair has not been verified in this review, so affected results need caution.
deep-swe · deep swe
- Source
- Hugging Face eval result
- Version
- Not reported
- Measurement date
- 2026-09-10
- Metric
- % · Higher is better
- Evaluation scope
- Registered model evaluation · DeepSeek-V4.1-Flash model card
Positions are withheld because at least one result lacks explicit independent provenance or a documented evaluation scope. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.
DeepSeek V4.1 Flash
deepseek/deepseek-v4.1-flash-20260910
74.2%
View modelEvidence Evidence class unreported