Baseten puts Mercury 2 in production, Augment reports 90% cost cut
- Source
- x.com
- Date
The fastest reasoning LLM is now in production on Baseten. Mercury 2 is a diffusion LLM, so it generates tokens in parallel and hits 1,000+ tokens/sec on @NVIDIAAI GPUs, speeds that used to require specialized hardware. @augmentcode is already using Mercury 2, cutting cost 90% and latency 82%. Proud to partner with the @baseten team to bring dLLMs to production.
Context
The reported Mercury 2 savings at Augment Code concern : summarizing a coding session so the can continue without carrying its entire history. Hosting provider Baseten says in its launch article that Augment assigned that step to Inception's model, reducing its cost by 90% and its duration from about 150 seconds to 27 seconds, an 82% reduction. The account illustrates how a supporting task can move to a different model while the main coding model handles other work. These are reported improvements to the compaction step, rather than the whole coding session.
- context compaction — Summarizing an agent's earlier conversation to free room in the context window so a long session can keep going.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
postInception ships Mercury Decide update with ZDR and 66k context
postInception launches Mercury Voice, a diffusion LLM for low-latency voice agents
postToday, we’re introducing Mercury 2.5, the most capable diffusion LLM on the…
postInception ships Mercury 2.5 Preview at roughly 1,100 tokens per second
Checking sign-in…
Loading comments…