← All IntelClip / EducationCPU scarcity: RL and agents are eating compute
From Fireside Chat: Simon Eskildsen on Building Turbopuffer · ≈38:54
“I think that GPUs will probably continue to be scarce.”
“I would assume that it gets a lot worse before it gets a lot better on the on the CPU side”
“even we, you know, we're selling to companies that we also fight for CPU with and against, right?”
What’s in it
- Reveals why CPUs, not GPUs, are now the real infra bottleneck
- Explains how RL training workloads are devouring CPU capacity
- Shows how applied AI failures loop back into more RL demand
Clip transcript
whatever they can do. I would think getting CPUs is should be easy. Is it? >> No, >> it's not anymore. Why? What's happening? Can you tell us about dynamics on on on the why and what you've learned? Yeah. So, I think that GPUs will probably continue to be scarce. Like, I don't know, maybe there's going to be some surplus. I I refuse to speculate too much about the macro, but I think as as RL is becoming a very very large amount of the workloads that needs a lot of CPUs. So, the labs are sucking up a lot of CPUs because you need CPUs to be like, okay, we need to like teach this model how to how to search. We need to teach it how to use GP. We need to teach it how to boot up bash. We need it needs to run real things and learn from that takes a lot of CPU. >> Um and so I think as we RL is consuming a lot of CPU and then also just all of the agents are running on CPUs, right? They need to do all kinds of very general purpose things on a CPU and so as as as the demand curve is sort of shifting to the right and it's becoming more and more applied and that feeds back into RL by the way, right? because as things become more applied like oh the models are not that good at CAD or ship building I don't know and then you know you have to spin up even more RL environments to do that so I think that's what we're seeing and so we're on the other end of that needing these CPUs we need a lot of NVME SSDs as well um and a lot of this right now is tied up in DRAM right of of where like >> you need a lot of that also for the GPU servers um but I would assume that it gets a lot worse before it gets a lot better on the on the CPU side um and I think even the big companies are fighting amongst each other, right, to get the allocations and even we, you know, we're selling to companies that we also fight for CPU with and against, right? It's uh it's it's it's really difficult and so you write things to try to make sure you get these CPUs as f fast as possible.
Comments
Sign in to comment.
Loading comments…