Open Weights Vs Closed Weights Models

Open-weight models are ones whose trained parameters are published so anyone can download, run and modify them, unlike closed models that are reachable only through a company's app or API. This October 2025 data insight from Epoch AI asks how far the best of them trail the most capable model available at any point (epoch.ai). The answer, on Epoch's own Epoch Capabilities Index (a composite score built from many benchmarks), is about three months on average, or roughly 7 index points, which Epoch says is similar to the distance between OpenAI's o3 and GPT-5.

The precise figures carry wide error bars: an average time gap of 3.5 months with a 90% confidence interval of 1.1 to 5.3 months, and an average score gap of 7 points with an interval of 0 to 14. Because index scores are noisy, Epoch counts an open model as having caught up when the difference from an earlier leader is not statistically significant. That is why DeepSeek's V2 is counted as matching GPT-4 after about 14 months despite scoring 125 against GPT-4's 126. The gap also moved around a lot and sometimes vanished: until o1-mini arrived, Llama 3.1-405B was rated level with the leading closed model, Claude 3.5 Sonnet.

Epoch added that the gap at publication probably looked wider than it was, since newer open models such as gpt-oss-120b and MiniMax-M2 did not yet have enough evaluations to be scored. A May 2026 follow-up using a bootstrap-based method found a slightly larger gap of four months and 8 points for January to May 2026.