ElevenLabs

ElevenLabs released Instant Soundtracks. The model scores uploaded footage with original music, places sound effects at the matching timestamps, and can take the original audio as a style reference. It is live in the Eleven apps and the API.

ElevenLabs launched Instant Soundtracks: upload a video and the model scores it, finds effect cues, and places generated sound effects on the timeline. It adds audio-as-reference and is available in the editor and via API parameters such as add_sfx.

ElevenLabs introduces Scribe v2 Realtime, a live speech-to-text model with under 150 ms latency, automatic language detection, text conditioning for reconnects and enterprise compliance, built for voice agents and live captioning.

ElevenLabs added References to Music v2: upload a 10-second to 5-minute track and generation matches its style, instrumentation and feel, alone or with a prompt. It ships in ElevenMusic, ElevenCreative and the API, with a copyright match check.

ElevenLabs details how it runs its own support on voice and chat agents: SOPs become procedures, webhook tools fetch account data, channel settings tune behavior, and a simulation suite gates changes. Reports 65% end-to-end resolution.

Patterns from forward-deployed engineers for authenticating callers to voice agents: session passthrough, knowledge-based checks, caller ID with a second factor, and OTP, all enforced by isolated sub-agents and deterministic workflow routing rather than LLM inference.

ElevenLabs launched agent testing: scenario simulations for tool calling, human handoffs, multi-step workflows and guardrails, with tests generated from real conversations and runnable in CI/CD pipelines.

ElevenAgents added SMS, Telegram, Intercom and Freshdesk channels alongside phone, web, Slack, WhatsApp and Zendesk. Agents get per-channel behavior tuning and channel-scoped simulations for testing before release. Telegram, Intercom and Freshdesk are alpha.

A postmortem on deploying a voice agent in product docs: defining success and escalation goals, evaluating calls with an LLM judge validated by human review, measuring knowledge-base hallucination, and noting where humans remain necessary.

ElevenLabs details Cue, a forced alignment model that times every character in one pass over recordings of any length. It reports lower error than open-source aligners across 29 languages and aligns an hour of audio in about half a second. Benchmarks are self-reported.

ElevenLabs cut self-serve API and agent pricing (Flash TTS 55% lower, Scribe v2 STT 45% lower, ElevenAgents 20% lower) and added a pay-as-you-go plan with a migration path for existing customers.

ElevenLabs rolls out multi-region Text to Speech serving. Requests route to US, Netherlands, or Singapore backends, cutting TTFB by roughly 50-200ms depending on location. An x-region header shows the serving region, and a US-only base URL opts out.

How two hackathon developers built agents on ElevenLabs conversational AI that recognize each other as AI mid-call and switch from speech to ggwave data-over-sound, and why that saves compute and improves precision.

Describes how to integrate frameworks like LangGraph or CrewAI behind a voice orchestrator via a Custom LLM, and how to separate internal conversational state from external system state so voice and text agents stay consistent.

A Node and TypeScript walkthrough for building a phone voice agent with Twilio Media Streams, streaming STT, an LLM, and TTS output in mu-law, including how to stream LLM tokens into synthesis to cut latency.

Explains why ElevenLabs limits are about concurrent requests, not per-minute counts, and how WebSockets change accounting. Walks through client-side concurrency pools, queue and 429 handling, multi-tenant fairness and monitoring with working TypeScript.

ElevenLabs extends its MCP server from agent management to generation. Assistants can produce speech, transcripts, dubs, music, sound effects, images and video through one OAuth connector with admin tool controls.

A builder details a multi-model ad generator: Gemini for ideas and scripts, Veo 2 for five-second scenes, ElevenLabs for voiceover and sound effects, FFmpeg for assembly, with notes on resilient error handling and vibe-coding it with Claude.

How timestamped speech-to-text works when the model emits typed tokens (word, spacing, audio_event) directly, why that beats forced alignment, and how to use it for captions, searchable archives and multichannel audio.

ElevenLabs launched Eleven Realtime v1 Mini and Max, speech-to-speech models covering 90+ languages and its voice library. The API is OpenAI Realtime compatible. Artificial Analysis reports 98.8% on Big Bench Audio for Max, with first speech in about 0.6 seconds.

ElevenLabs adds Experiments to its agent platform. Teams create versioned variants, route a controlled share of live traffic to them, compare CSAT, containment, conversion, latency and cost, and migrate traffic to the winner.

ElevenLabs released a Finetunes Music API that lets products train the Eleven Music model on a user's own tracks. Finetunes are private per user, work with the existing compose endpoints, and uploads pass a third-party content ID check.

ElevenLabs' sales team describes building voice roleplay coaches on ElevenAgents. Buyer personas reveal information in tiers based on how sharp the rep's questions are, with coaching logic in a visual workflow builder and product docs in each agent's knowledge base.

ElevenLabs explains multimodal input in ElevenAgents: images and PDFs arrive as native model messages on web, mobile, and WhatsApp. Covers file-backed versus inline representation, setup, and carrying context across sessions.