TurnBench Scores Whether Voice AI Knows When You're Done Talking
- Source
- x.com
- Date
To craft for conversation, timing is everything. Natural conversation is a social dance, and timing matters just as much as tone and delivery. Most voice AI doesn't do this well yet. So we built TurnBench: an open benchmark that measures whether voice AI knows when your turn is really over, and when an "mm-hmm" is agreement, not an interruption. Learn more about how we built this on our blog. https://t.co/idk8fzJH0W
Available now at https://t.co/2FfCFTDFQM, it features a public leaderboard, an interactive conversation viewer, and self-serve scoring on the dev set.
Context
Sesame's TurnBench tests whether voice systems recognize when to speak and when to yield. Its benchmark description separates a genuine turn ending from a pause, and a real interruption from a brief acknowledgment such as “mm-hmm.” Scoring counts both detected events and false alarms, alongside response timing. That matters because a system could react quickly by treating every silence or sound as a turn change, yet repeatedly interrupt the conversation. The uses studio-recorded English conversations between two speakers, so its scores describe performance under those conditions.
- benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
Checking sign-in…
Loading comments…