New Streaming Speech To Text Benchmark Aa Wer Streaming
Source
artificialanalysis.ai
Author
Artificial Analysis articles
Date
Why it matters
Voice AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → builders can choose streaming STT by measured accuracy and latency after end of speech, such as Deepgram Flux at about 0.02s versus Cartesia and ElevenLabs at lower WER.
Key takeaways · AI-distilled
AA-WER streamingSending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.Full definition → pairs word error rate with latency measured from Silero VAD-detected end of speech, at two points: the first final transcript and the first transcript event of any kind. It reuses the roughly 8 hours of audio from the offline AA-WER v2.0 benchmarkA standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.Full definition →.
On first final transcripts, Cartesia Ink-2 with semantic endpoints was most accurate (3.59% WER, 0.21s), with ElevenLabs Scribe v2 Realtime close behind (3.64%, 0.14s). Deepgram Flux was fastest at about 0.02s but at 7.36% WER.
Waiting for the final transcript improved accuracy by only about 0.7 points on average; most models already hit final accuracy at the first partial. Voxtral Mini Transcribe Realtime (9.6% to 5.3%) and GPT Realtime Whisper (7.5% to 5.1%) were exceptions.
No model led every dataset: ElevenLabs led the private AA-AgentTalk set, AssemblyAI and Google led VoxPopuli, and Cartesia led Earnings22. Prices ranged from $2 to $17 per 1,000 minutes, with Soniox cheapest but least accurate at 11.9% WER.
Terms in this piece · Glossary
streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
