Kimi K3 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing

The leading coding model can change when a test permits retries. In Together AI's analysis of 113 DeepSWE tasks, OpenAI's GPT-5.6 Sol led Moonshot AI's Kimi K3 on single-attempt performance, 72.7% to 68.5%. With four attempts, Kimi's reported coverage was higher, 89.4% to 85.8%.

Four-attempt coverage counts a task when at least one attempt passes the tests. It measures whether a model can reach a working solution within that budget. Making use of that advantage requires additional attempts and a way to recognize which result works, so retry performance answers a different deployment question from first-attempt reliability.

“GPT-5.6 Sol edges Kimi K3 on single-shot quality, but Kimi wins on pass@k with k > 1 and costs 64% less per completed task.”

Together AI

“Kimi K3 is far cheaper: \$4.65 per rollout vs \$8.37, and 2.8x more solved tasks per dollar.”

Together AI

“The practical answer is about 85.6%: run Kimi K3 first and escalate to Sol only when the test suite rejects the result. That beats either model alone and even a perfect one-shot router (83.4%), because the cascade gives hard tasks two independent attempts instead of committing to one.”

Together AI

“The hard ceiling for this pair is 95.6%. Five of the 113 tasks are solved by neither model across all eight combined attempts. Past 95.6% you need a third model, not a better router.”

Together AI

“Sol breaks the repo's existing tests in 20% of failures - this is pretty consistent with other GPT models actually.”

Together AI