The fastest reasoning LLM is now in production on Baseten.
Mercury 2 is a diffusion LLM, so it generates tokens in parallel and hits 1,000+ tokens/sec on @NVIDIAAI GPUs, speeds that used to require specialized hardware.
@augmentcode is already using Mercury 2, cutting cost 90% and latency 82%.
Proud to partner with the @baseten team to bring dLLMs to production.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
Why it matters
Another production serving path for a diffusion LLMA large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.Full definition →, with a named customer reporting 90% lower cost and 82% lower latency after switching workloads to Mercury 2.
