Vibeleaderboard
← Benchmark guide

Model results

SWE-bench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Reviewed recommendation and limits

Use Verified as a historical shared-agent reference. Start with the locked public SWE-bench Pro V2 protocol for contemporary repository issue repair.

Historical Verified with mini-SWE-agent; preserve each imported suite and harness. Imported groups below may use different protocols; this review does not certify each run. OpenAI stopped reporting Verified after finding task defects and memorized solutions. Its 59.4% defect finding concerns 138 selected difficult tasks, not the full 500-task set. Imported Epoch runs have their own protocol.

Publisher results OpenAI audit of SWE-bench Verified

Evaluation groups

Open a group to see its exact protocol and scores. Different configurations remain separate.

Model rankings

Open a group to compare scores within its reported protocol. Positions do not establish statistical significance.

Other reported results

These reports have no model ranking. Open a group for its evidence and the reason positions are withheld.

  • Beam 1.0 501B A23Bunversioned

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-10-07

    Scope · Registered model evaluation · dataset SWE-bench/SWE-bench_Verified · task swe_bench_%_resolved · Bea…

    Registered model evaluation · dataset SWE-bench/SWE-bench_Verified · task swe_bench_%_resolved · Beam 1.0 model card

  • SWE-bench_Verified · swe bench % resolvedVersion not reported

    Hugging Face eval result

    2 models with scores · Positions withheld

    Measurement date 2026-08-31

    Scope · Registered model evaluation · Model Card

  • Nemotron 3.5 LightningVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-08-11

    Scope · Registered model evaluation · NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 model card

  • Muse Glimmer 30BVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-08-10

    Scope · Registered model evaluation · Muse Glimmer-30B model card

  • Inkling SmallVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-07-31

    Scope · Registered model evaluation · Model Card

  • InklingVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-07-19

    Scope · Registered model evaluation · Model Card

  • Hy3Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-07-06

    Scope · Registered model evaluation · Model Card

  • Laguna XS 2.1Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-07-02

    Scope · Registered model evaluation · Model Card

  • MiniMax M3Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-06-23

    Scope · Registered model evaluation · MiniMax-M3 model card

  • North Mini CodeVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-06-09

    Scope · Registered model evaluation · North Mini Code Model Page

  • Nemotron 3 UltraVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-06-03

    Scope · Registered model evaluation · Model Card

  • MiMo-V2.5-ProVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-04-27

    Scope · Registered model evaluation · Model Card

  • SWE-bench_Verified · swe bench % resolvedVersion not reported

    Hugging Face eval result

    2 models with scores · Positions withheld

    Measurement date 2026-04-24

    Scope · Registered model evaluation · Model Card

  • Qwen3.6 27BVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-04-22

    Scope · Registered model evaluation · Model Card

  • Kimi K2.6Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-04-20

    Scope · Registered model evaluation · Model Card

  • Qwen3.6 35B A3BVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-04-16

    Scope · Registered model evaluation · Model Card

  • Hy3 previewVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-04-14

    Scope · Registered model evaluation · Model Card

  • Trinity Large ThinkingVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-04-02

    Scope · Registered model evaluation · Model Card

  • GLM 5Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-03-24

    Scope · Registered model evaluation · SWE-Bench official evaluation

  • GLM 4.7 FlashVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-03-18

    Scope · Registered model evaluation · Model Card

  • SWE-bench_Verified · swe bench % resolvedVersion not reported

    Hugging Face eval result

    4 models with scores · Positions withheld

    Measurement date 2026-03-17

    Scope · Registered model evaluation · Model Card

  • Qwen3 Coder NextVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-03-16

    Scope · Registered model evaluation · Qwen3-Coder-Next technical report

  • MiniMax M2.5Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-03-10

    Scope · Registered model evaluation · SWE-Bench official evaluation

  • Kimi K2.5Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-03-02

    Scope · Registered model evaluation · SWE-Bench official evaluation

  • DeepSeek V3.2Version not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-02-28

    Scope · Registered model evaluation · SWE-Bench official evaluation

  • Step 3.5 FlashVersion not reported

    SWE-bench_Verified · swe bench % resolved · Hugging Face eval result

    Single-model result · Positions withheld

    Measurement date 2026-02-11

    Scope · Registered model evaluation · Step 3.5 Flash Paper

  • SWE-bench_Verified · swe bench % resolvedVersion not reported

    Hugging Face eval result

    4 models with scores · Positions withheld

    Measurement date 2026-02-03

    Scope · Registered model evaluation · Model Card

  • SWE-bench_Verified · swe bench % resolvedVersion not reported

    Hugging Face eval result

    2 models with scores · Positions withheld

    Measurement date 2025-08-05

    Scope · Registered model evaluation · GPT-OSS Model Card

  • SWE-bench VerifiedVersion not reported

    Hugging Face OpenEvals

    17 models with scores · Positions withheld

    Measurement date not reported

    Scope · Latest published official benchmark result aggregated by OpenEvals · result date not supplied

  • Xing4.0-29B-A4BVersion not reported

    SWE-bench Verified · XingChen-AGI model card

    Single-model result · Positions withheld

    Measurement date not reported

    Scope · Publisher model-card report; measurement date and metric unit unreported. SWE-agent; temperature 1.0…

    Publisher model-card report; measurement date and metric unit unreported. SWE-agent; temperature 1.0, top_p 0.95, repetition_penalty 1.05, 210K context. Run count unspecified.

SWE-bench model results | VibeLeaderboard