The claimed savings come from idle GPUs, not raw math. The model searched its own forward pass for work that could be precomputed, skipped, or parallelized — cutting memory movement and synchronization stalls, for 20% lower end-to-end serving cost.
The kernels were rewritten in Triton and Gluon, two GPU programming languages OpenAI itself maintains. Training the model to be good at your own languages, then pointing it at your own serving stack, is a vertically integrated loop.
Anthropic's Haiku 4.5 at $1/$5 per million tokenThe chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.Full definition → used to be price-matched with OpenAI's cheap tier. After the cut, the same input tokens cost a fifth as much — enough to make people re-pick the default model for high-volume work.
Terms in this piece · Glossary
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
If you're choosing a budget LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition → tier, GPT-5.6 Luna at $0.20/$1.20 per million tokens is now cheaper than Gemini 3.1 Flash-Lite and Claude Haiku 4.5, and the underlying cost cuts come partly from AI-driven kernel optimization rather than just margin compression — a signal of where inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → cost curves are headed.
Key quotes
“We also used GPT‑5.6 Sol to optimize the model’s forward pass: the computation that transforms inputs into next-token predictions.”
“With Codex, GPT‑5.6 Sol autonomously rewrote and optimized our production kernels, the core code that executes the mathematical operations that make up the model.”
“That Luna price drop completely changes the landscape with respect to lower priced models.”
