"An endless demand for compute" | Jonathan Ross, founder of Groq
Source
Julia Turc
Author
Julia TurcTop Viber
Date
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Why it matters
inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → economics decide what AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → workloads are viable in production, and this explains the hardware tradeoffs behind them.
Key quotes
“As long as there are unsolved problems in civilization, we will have a need for more compute.”
Jonathan Ross
“Why that's leading to an absolute explosion in the usage is AI using AI using AI.”
Jonathan Ross
“My response was how hard could it be? Turns out it was very hard but we didn't know so jumped in.”
Jonathan Ross
“If you can come up with the right question, AI can go answer it for you.”
Jonathan Ross
“The more compute we provide for the world, the more compute people are going to want.”
