A Brain Too Big to Carry — On-Device vs Datacenter Inference
Source
Ivan Chiam
Author
Ivan Chiam
Date
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
Explains why robot AI models can't simply scale like cloud LLMs: real-time latency limits and fixed per-unit manufacturing costs mean the model has to be designed around what's affordable to build and run on-device.
