When Running Your Own AI Models Beats Paying for APIs - Steven Latré, imec

Paying per token for a coding agent is simple, but bills grow with every task, so some teams consider running open-weight models on GPUs they own or rent. Imec, the Belgian semiconductor research center, measured that trade-off by running coding agents on 64 tasks from SWE-Bench Pro, a benchmark of real software-engineering problems, across self-hosted hardware and commercial APIs. Its written report on the study counts cost per completed task rather than raw speed, and it concludes that GPUs are worth buying for control, privacy or latency rather than for cost alone. In this AI Engineer talk, Steven Latré, who leads AI at imec, presents imec's benchmarking study. No transcript was available, so this explanation relies on the talk's description, which cites the same benchmark and break-even figures as the report.

The deciding number is utilization, the share of time the hardware is actually doing work. An owned GPU costs the same whether it is busy or idle, so its price per task falls only as it handles more tasks. Imec's report compares that against the API each setup would replace. A four-GPU H200 server running DeepSeek's V4-Flash model had to stay about 89% loaded to beat DeepSeek's own API, which cost $2.13 for the whole task set, partly because that API caches the repeated input coding agents send. An eight-GPU B200 machine running GLM-5.2 needed only about 15% utilization to beat Anthropic's Opus 4.8 API, which cost $98 for the same tasks. The talk's description summarizes the spread as break-even utilization ranging from 89% on small systems to about 15% on 8x B200 machines.

Capacity is the other constraint, and the talk's description says concurrency matters even more than accuracy. In the report, a single H200 running a small Qwen model handled about 32 simultaneous agent sessions comfortably, while the eight-GPU B200 machine running GLM-5.2, the setup that matched Opus 4.8's accuracy on these tasks, could comfortably serve only about eight users, and at that load was already about three times slower than the Claude Code baseline running Opus 4.8. So the big machine is cheap per task when compared with a frontier API, but it serves only a small team at a usable speed. A setup that serves many developers at once tends to keep GPUs busy, which is the condition the numbers reward. The talk's description adds one finding the July report does not cover: across three coding harnesses, the software that drives the model, the right choice could cut GPU time in half for the same result. Without a transcript, that claim could not be checked.

The report lists its own limits. Its costs leave out power and maintenance, its prices are July 2026 advertised rates, and its serving engines were not tuned for the workload. For a team weighing a move off APIs, the study does not give a single verdict. It shows that the answer turns on the specific API being replaced and on whether enough developers share the hardware. Revolut's account of self-hosting for its engineering organization is one company reporting that the trade paid off at its scale.