Model results
NL2Repo-Bench
Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.
Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.
Evaluation groups
Open a group to see its exact protocol and scores. Different configurations remain separate.
No model rankings are available in this snapshot. Reported results remain available below.
Other reported results
These reports have no model ranking. Open a group for its evidence and the reason positions are withheld.
- DeepSeek V4.1 Flashunversioned
NL2Repo-Bench · DeepSeek launch chart (vendor-reported)
Single-model result · Positions withheld
Measurement date 2026-09-10
Scope · Vendor-reported; launch chart; conflicting evidence: model card reports 64.0; unresolved
- DeepSeek V4.1 Flashunversioned
NL2Repo-Bench · DeepSeek model card (vendor-reported)
Single-model result · Positions withheld
Measurement date 2026-09-10
Scope · Vendor-reported; reasoning effort 100; Score; temperature 1.0, top_p 0.95; DeepSeek Harness Minimal;…
Vendor-reported; reasoning effort 100; Score; temperature 1.0, top_p 0.95; DeepSeek Harness Minimal; 1M context; conflicting evidence: launch chart reports 65.4, model card reports 64.0; unresolved