Intel / article

Voice Agent Latency Optimization

Why it matters

Gives a concrete latency budget for cascaded voice agents and shows endpointing and time-to-first- are the largest items. Overlapping stages and recover most of the budget, so you know where to optimize first.

Terms in this piece · Glossary

LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.