The Pulse #191: a new trend of CPU shortages

Gergely Orosz opens this issue of The Pulse with a claim about the next hardware bottleneck: after AI companies drove a shortage of GPUs and then of memory, he says the industry is now seeing a CPU shortage, because AI agents use a lot more CPU through tool usage. His practical conclusion, in the issue's subtitle, is that teams running compute-intensive services should reserve more compute now if they expect to need it later.

The reason tool use lands on CPUs is the split between thinking and doing. The model that decides what an agent should do next runs on GPUs, but the actions it takes, such as running code, calling tools and working inside sandboxes, run as ordinary software. NVIDIA describes the same division in its post on agent fleets: GPUs run the models while CPUs handle orchestration, tool execution and sandboxed computation. That post, which pitches NVIDIA's Vera CPU as a chip designed for agent workloads, is covered in earlier Intel.

Only the opening summary of Orosz's issue is public. The evidence behind the shortage claim, including which providers or workloads are affected and how scarce capacity has become, is in the paid portion, so the claim and the advice can be read here but not checked.

“New trend of CPU shortages: after a GPU shortage and memory shortage driven by AI companies, we’re now experienceding a CPU shortage, thanks to AI agents using a lot more CPU with tool usage.”

Gergely Orosz

“For compute-intensive services: it’s worth reserving more compute now.”

Gergely Orosz