Where Does a Robot Think – On-Device vs Datacenter Inference

SemiAnalysis argues that many robots will need to move their heaviest model computation off the robot. A datacenter-class GPU draws far more power than a humanoid robot's battery and cooling can support, and in the hierarchical designs it focuses on, a large, slow planning model updates only a few times a second while a small, fast action model turns the plan into motor commands hundreds of times a second. The planning layer can therefore tolerate network delay that the control loop cannot.

In its test, SemiAnalysis ran NVIDIA's DreamZero, a 14-billion-parameter world action model, on one B300 GPU, serving robots one after another because the model keeps the GPU fully busy even for a single robot. It assumed a 10-millisecond network round trip with the GPU on site, and reports that the GPU handled seven robots with a 99th-percentile response time of 1.16 seconds, under the 1.6 seconds each inference buys before a robot runs out of instructions. The run used a single denoising step without the retraining recipe of the setup it imitates, so SemiAnalysis says task quality is unvalidated. Meeting the timing budget therefore does not establish that the resulting robot actions work well.

“The model comes first; the hardware serves it.”

“Jitter is the biggest enemy.”

“So the real question is not whether robots can get wafers. It is silicon efficiency: for a given fleet of robots, which approach consumes less leading-edge silicon?”

“Serve a VLA fleet with continuous batching; serve a video WAM one robot at a time.”