Inception positions Mercury 2.5 for applications that make many short model calls before returning an answer. Its launch article explains how one search request can trigger dozens of model calls to plan the search, rewrite queries, rerank results, structure facts, summarize sources and check the answer. Faster generation can reduce the repeated waits in that sequence. The company reports a speed of 1,107 tokens per second for Mercury 2.5 on widely available NVIDIA GPUs, but that is a generation-rate claim from the vendor, not an end-to-end response time for an entire search or voice application.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
context compaction — Summarizing an agent's earlier conversation to free room in the context window so a long session can keep going.
