Model results

SWE-rebench

Choose a shared evaluation protocol to compare model scores. Versions, sources and configurations stay separate.

Atlas snapshot captured 10/19/2026, 10:20:28 AM UTC. This is an import timestamp, not the evaluation date. Coverage may be incomplete.

Official source

Evaluation groups

Open a group to see its exact protocol and scores. Different configurations remain separate.

No model rankings are available in this snapshot. Reported results remain available below.

Other reported results

These reports have no model ranking. Open a group for its evidence and the reason positions are withheld.

Claude Fable 52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Fable 5 [high]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 64.5% with SEM 1.41 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas anthropic/claude-fable-5; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-09 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Claude Fable 52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Fable 5 [high]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 64.5% with SEM 1.41 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas anthropic/claude-fable-5; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-09 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Claude Opus 52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Opus 5 [high]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 63.4% with SEM 1.35 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas anthropic/claude-opus-5; exact execution backend and API request model identifier unverified; publisher model release date 2026-07-24 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Claude Opus 52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Opus 5 [high]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 63.4% with SEM 1.35 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas anthropic/claude-opus-5; exact execution backend and API request model identifier unverified; publisher model release date 2026-07-24 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Claude Sonnet 52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Sonnet 5 [high]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 56.8% with SEM 0.94 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas anthropic/claude-sonnet-5; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-30 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Claude Sonnet 52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Sonnet 5 [high]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 56.8% with SEM 0.94 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas anthropic/claude-sonnet-5; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-30 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
DeepSeek V4 Pro 04232026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration DeepSeek-V4 Pro [high]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 40.2% with SEM 1.29 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas deepseek/deepseek-v4-pro-20260423; exact execution backend and API request model identifier unverified; publisher model release date 2026-04-24 is not a measurement date or proof of the Atlas dated execution; original V4 Pro declaration with high effort, not the August 13 successor; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-10-19
DeepSeek V4 Pro 04232026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration DeepSeek-V4 Pro [high]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 40.2% with SEM 1.29 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas deepseek/deepseek-v4-pro-20260423; exact execution backend and API request model identifier unverified; publisher model release date 2026-04-24 is not a measurement date or proof of the Atlas dated execution; original V4 Pro declaration with high effort, not the August 13 successor; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-10-19
GLM 5.22026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration GLM-5.2 [high]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 62.9% with SEM 1.19 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas z-ai/glm-5.2-20260616; exact execution backend and API request model identifier unverified; publisher model release date 2026-05-16 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
GLM 5.22026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration GLM-5.2 [high]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 62.9% with SEM 1.19 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas z-ai/glm-5.2-20260616; exact execution backend and API request model identifier unverified; publisher model release date 2026-05-16 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
GPT-5.6 Luna2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration GPT-5.6 Luna [medium]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 43.6% with SEM 1.47 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas openai/gpt-5.6-luna-20260709; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-26 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
GPT-5.6 Luna2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration GPT-5.6 Luna [medium]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 43.6% with SEM 1.47 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas openai/gpt-5.6-luna-20260709; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-26 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
GPT-5.6 Sol2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration GPT-5.6 Sol [medium]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 62.3% with SEM 1.83 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas openai/gpt-5.6-sol-20260709; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-26 is not a measurement date or proof of the Atlas dated execution; declared Sol result does not attest to Sol Pro execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
GPT-5.6 Sol2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration GPT-5.6 Sol [medium]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 62.3% with SEM 1.83 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas openai/gpt-5.6-sol-20260709; exact execution backend and API request model identifier unverified; publisher model release date 2026-06-26 is not a measurement date or proof of the Atlas dated execution; declared Sol result does not attest to Sol Pro execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Grok 4.52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Grok 4.5 [high]; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 63.8% with SEM 0.60 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas x-ai/grok-4.5-20260708; exact execution backend and API request model identifier unverified; publisher model release date 2026-07-08 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Grok 4.52026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Grok 4.5 [high]; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 63.8% with SEM 0.60 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas x-ai/grok-4.5-20260708; exact execution backend and API request model identifier unverified; publisher model release date 2026-07-08 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
MiMo-V2.5-Pro2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration MiMo V2.5 Pro; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 46.5% with SEM 0.54 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas xiaomi/mimo-v2.5-pro-20260422; exact execution backend and API request model identifier unverified; publisher model release date 2026-03-22 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-09-10
MiMo-V2.5-Pro2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration MiMo V2.5 Pro; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 46.5% with SEM 0.54 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas xiaomi/mimo-v2.5-pro-20260422; exact execution backend and API request model identifier unverified; publisher model release date 2026-03-22 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-09-10
MiniMax M32026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration MiniMax M3; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 47.2% with SEM 1.13 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas minimax/minimax-m3-20260531; exact execution backend and API request model identifier unverified; publisher model release date 2026-05-31 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
MiniMax M32026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration MiniMax M3; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 47.2% with SEM 1.13 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas minimax/minimax-m3-20260531; exact execution backend and API request model identifier unverified; publisher model release date 2026-05-31 is not a measurement date or proof of the Atlas dated execution; publisher flags potential contamination because its model release date follows the task-window start; this is not proof of training contamination; result date not supplied; source captured 2026-09-10
Qwen3.5-35B-A3B2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Qwen3.5-35B-A3B; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 17.1% with SEM 1.07 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas qwen/qwen3.5-35b-a3b-20260224; exact execution backend and API request model identifier unverified; publisher model release date 2026-02-15 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-10-19
Qwen3.5-35B-A3B2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Qwen3.5-35B-A3B; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 17.1% with SEM 1.07 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas qwen/qwen3.5-35b-a3b-20260224; exact execution backend and API request model identifier unverified; publisher model release date 2026-02-15 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-10-19
Qwen3.6 27B2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Qwen3.6-27B; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 31.2% with SEM 1.68 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas qwen/qwen3.6-27b-20260422; exact execution backend and API request model identifier unverified; publisher model release date 2026-03-22 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-09-10
Qwen3.6 27B2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Qwen3.6-27B; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 31.2% with SEM 1.68 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas qwen/qwen3.6-27b-20260422; exact execution backend and API request model identifier unverified; publisher model release date 2026-03-22 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-09-10
Qwen3.6 35B A3B2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; con…
pass-at-5; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Qwen3.6-35B-A3B; tools interaction mode; minimal ReAct framework; pass@5 across five full-benchmark runs; published resolved mean 24.7% with SEM 0.79 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas qwen/qwen3.6-35b-a3b-20260415; exact execution backend and API request model identifier unverified; publisher model release date 2026-03-14 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-09-10
Qwen3.6 35B A3B2026-05-15_2026-07-01
SWE-rebench · SWE-rebench (reviewed publisher results)
Single-model result · Positions withheld
Measurement date not reported
Scope · resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track;…
resolved-rate; 111 problems from 65 repositories; task window 2026-05-15 to 2026-07-01; model track; configuration Qwen3.6-35B-A3B; tools interaction mode; minimal ReAct framework; mean resolved rate over five full-benchmark runs; published resolved mean 24.7% with SEM 0.79 percentage points; SEM is standard error of the mean, not a confidence interval or standard deviation; publisher context policy 128K tokens unless the model supports less; developer-recommended default generation settings; declared-model match to Atlas qwen/qwen3.6-35b-a3b-20260415; exact execution backend and API request model identifier unverified; publisher model release date 2026-03-14 is not a measurement date or proof of the Atlas dated execution; publisher does not flag potential contamination for this window; this does not prove decontamination; result date not supplied; source captured 2026-09-10