Vibeleaderboard
← All Intel
Intel / article

Scribe V3

Source
elevenlabs.io
Author
ElevenLabs editorial sitemap
Date

Context

ElevenLabs has released two speech-to-text models: Scribe v3 for recorded audio and Scribe v3 Live for real-time audio such as voice agents. The headline figures in the announcement come from ElevenLabs' own testing. Scribe v3 makes 27% fewer word errors than Scribe v2, averaged over 102 languages, and Scribe v3 Live cuts semantic word error rate by 43% compared with Scribe v2 Realtime. Word error rate counts the words a model gets wrong, drops or adds against a reference transcript. Semantic word error rate counts an error only when it changes the meaning, which ElevenLabs argues is the fairer measure for models, because they format numbers, punctuation and capitals differently while text is still arriving. On ordinary word error rate the Live improvement is 31%.

The changes aimed at conversations concern who is speaking and when they have finished. Diarization, labeling who said what, comes to ElevenLabs' real-time transcription for the first time with Scribe v3 Live, and ElevenLabs reports a 3.1% diarization error rate on the English part of its own dataset of meetings, calls and voice notes, against 5.9% to 7.3% for the other real-time models it tested. Turn detection decides when a speaker is done. A silence timer cuts off callers who pause mid-sentence, so Scribe v3 Live also uses the conversation's , letting a caller pause partway through reading out a number. ElevenLabs says the Dutch energy supplier Eneco tested Scribe v3 Live in its voice and that, in real customer calls, the agent recognized postcodes correctly on the first try 40% more often.

None of these results has been independently measured. Accuracy was scored on an internal version of the FLEURS , which consists of short, clean read speech, and diarization on a dataset ElevenLabs built and says it will open-source soon. ElevenLabs also scored each language separately before averaging and warns that results from pooled clips are not directly comparable. For developers already on ElevenAPI, upgrading is a change of model ID to `scribe_v3` or `scribe_v3_live`.

Key takeaways · AI-distilled
  • Scribe v3 Live predicts turn ends from conversational context as well as audio, so a caller who pauses mid-number is not cut off; ElevenLabs says silence thresholds alone force a choice between interrupting people and sounding slow.
  • Keyterm prompting accepts up to 1,000 known terms, and Scribe v3 Live adds a keyterm library holding thousands of possible terms that the model consults only when it hears a likely match.
  • On English real-world audio, ElevenLabs reports a 3.1% realtime diarization error rate for Scribe v3 Live versus 5.9% to 7.3% for other realtime models it tested, scored after aligning transcripts so missed words are not counted.
  • ElevenLabs frames the stakes: a one-hour meeting is about 9,000 words, so the gap between 4% and 2% word error is 180 wrong words, and errors cluster in names, numbers and jargon.
  • In pre-launch tests on real calls, energy supplier Eneco's voice agent recognized Dutch postcodes correctly on the first try 40% more often and house numbers 15% more often with Scribe v3 Live, according to ElevenLabs.
Terms in this piece · Glossary
  • streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
  • context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Read the source elevenlabs.io
More from ElevenLabs editorial sitemap
Recommended reads
Comments

Checking sign-in…

Loading comments…