AssemblyAI ships Universal-3.6 Pro Realtime for voice agents

AssemblyAI has released Universal-3.6 Pro Realtime, a streaming speech-to-text model. It transcribes a caller's audio as the audio arrives, so a voice agent can respond during the conversation. The company's launch post says the model leads two independent benchmarks that run every streaming model on the same audio. On Daily's open Pipecat speech-to-text benchmark, it sits on the Pareto frontier, meaning no other tested model beats it on all of the benchmark's competing measures at once. On Coval's live leaderboard of real voice-agent audio, it has the lowest word error rate of any streaming model.

AssemblyAI's argument is that voice agents fail at moments that other benchmarks skip. A one-word reply such as "No" or "Nah" decides what the agent does next, and the company says the model hears these replies correctly 98.5% of the time, including in loud rooms and on phone lines. It also says the model keeps background voices such as a TV out of the transcript and follows callers who switch among its 32 supported languages mid-sentence. Another feature, entity-aware endpointing, decides when the caller has finished speaking but keeps the turn open while someone is still reading out a phone number. AssemblyAI says the model was trained on tens of thousands of hours of real voice-agent and telephony conversations.

All of these claims come from AssemblyAI. The post's text does not give the benchmark scores, name the competing models, or explain how the 98.5% figure was measured. Its attached image may carry more detail but was not available, so the size of the lead cannot be judged from the text.