Model results

NL2Repo-Bench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source

← All evaluation groups

NL2Repo-Bench

Source: DeepSeek model card (vendor-reported)
Version: unversioned
Measurement date: 2026-09-10
Metric: points · Higher is better
Evaluation scope: Vendor-reported; reasoning effort 100; Score; temperature 1.0, top_p 0.95; DeepSeek Harness Minimal; 1M context; conflicting evidence: launch chart reports 65.4, model card reports 64.0; unresolved

Positions are withheld because at least one result lacks explicit independent provenance or a documented evaluation scope. Missing metadata is shown as unreported; matching imported metadata is not independent verification of the protocol.

DeepSeek V4.1 Flash
deepseek/deepseek-v4.1-flash-20260910
64 points
View modelEvidence model-card-reported