Vibeleaderboard
← All Intel
Intel / article

AssemblyAI-STT-3.7 splits transcription into three models

Source
assemblyai.com
Author
AssemblyAI
Date

Context

AssemblyAI, which sells speech-to-text through an API, released AssemblyAI-STT-3.7 as three models because, in its account, live transcripts, voice agents and recordings go wrong in different ways, and a model tuned for one gives something up on the others (announcement). STT-3.7-Realtime transcribes live meetings and calls, STT-3.7-Realtime- listens for a voice agent that has to answer a caller, and STT-3.7 handles recorded files. Each is built around the error that matters most for its audio.

The agent model shows the reasoning most clearly. A voice agent must tell a caller's quiet yes from a television in the background, and the two look alike to a speech model: both are short and low in energy. Filters that reject one tend to reject the other. AssemblyAI's research post says the new model reads each turn with the conversation so far, including the agent's last question and earlier audio of the caller, and was trained on calls where background voices stay in the audio but never in the target transcript. On 12,460 scripted test calls, compared with its previous realtime model, Universal-3.6 Pro Realtime, background words fell 47% and dropped short replies 38%. On a separate set of 300 number-heavy recordings, number errors fell 80%. The final transcript now arrives 86 milliseconds after the caller stops, down from 172.

These results come from AssemblyAI's own tests on its own scripted call set, run on each provider at default settings. On those calls it reports the lowest word error rate of 26 models and an entity error rate, the share of names, numbers, dates and email addresses transcribed wrong, of 13.2%, against 15.2% for Soniox Realtime v6, 16.9% for ElevenLabs Scribe v3 Live and 25.1% for Deepgram Nova-3. Two outside benchmarks it cites point the same way: Voice Arena, which uses a private set no vendor can train against, ranks it the top streaming model for US English, and Speko places it ahead of five rivals on accuracy and speed.

For recordings, the priority is who said what. Diarization, the step that assigns each word to a speaker, runs on a new model that weighs its own confidence word by word, so a quick yeah lands on the right person without breaking up sentences. AssemblyAI's pre-recorded research post reports 36% lower diarization error than Universal-3.5 Pro at launch, and on its internal comparison a 10.7% error rate against 12.0% for ElevenLabs Scribe v3, with other providers at 15% or higher. Coverage grows from 18 to 32 languages, and the model is less likely to write accented English in another script: stray foreign characters fell 85%.

The live model changes when speaker labels can be trusted. They are final as each turn ends and nearly as accurate as labels from a recording, with no later revision. At $0.30 an hour, against $0.45 for the agent model and $0.21 for recordings, AssemblyAI pitches it at meeting notetakers that still send live audio through the recorded pipeline. It does not judge when a caller has finished speaking, so AssemblyAI directs any system that waits on that decision, such as a voice agent, to the agent model.

Key takeaways · AI-distilled
  • AssemblyAI split transcription into three models by use: STT-3.7-Realtime for live meetings and calls at $0.30/hr, STT-3.7-Realtime-Agent for voice agents at $0.45/hr, and STT-3.7 for recordings at $0.21/hr. It says a model tuned for one use gives up something on the others.
  • On the live-conversation model, speaker labels are final the moment a turn ends, with no revision pass, and AssemblyAI says they are nearly as accurate as pre-recorded labels. Its pitch is that meeting notetakers no longer need to batch live audio through the recorded pipeline.
  • The voice-agent model returns a final transcript 86 ms after speech ends, and only 1.0% of words change between partial and final, down from 3.2%. On AssemblyAI's 12,460 English voice-agent calls, number errors fell 80% against Universal-3.6 Pro Realtime.
  • AssemblyAI reports that the agent model adds 47% fewer words while the caller is silent, aimed at background talkers such as kids shouting a drive-thru order. Its competitor comparisons are AssemblyAI's own runs of each vendor's API at default settings on the same recordings.
  • Switching changes defaults: STT-3.7 no longer falls back to Universal-2 unless it is listed, and it keeps fillers and false starts unless disfluencies is turned off. Universal-3.5 Pro Realtime retires on February 1, 2027.
Terms in this piece · Glossary
  • AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
  • streaming — Sending a model's response token by token as it is generated, so the reader sees text immediately instead of waiting for the whole answer.
Read the source assemblyai.com
More from AssemblyAI
Recommended reads
Comments

Checking sign-in…

Loading comments…