Unity Sdk Release

ElevenLabs, the voice AI company, has released a Unity SDK that lets a character in a Unity game hold a spoken conversation with the player. Unity is a widely used game engine, and the SDK, which ElevenLabs calls the ElevenAgents Unity SDK, connects a scene to an ElevenLabs voice agent that the developer first creates in the ElevenLabs dashboard, where the agent's prompt and voice are kept. During a session the SDK listens through the player's microphone, runs the conversation through the ElevenLabs Conversation API, and plays the agent's replies through an AudioSource, the Unity component that plays sound from an object in the scene. In the GettingStarted sample that ships with the SDK, a trigger zone around a character opens a session when the player walks in and closes it when they walk out, which takes one call to start and one to end.

Most of the post is about making that voice behave like part of the game rather than a separate chat window. The voice plays in 2D by default, meaning it is not placed anywhere in the scene, but a developer can hand the session the character's own AudioSource so the voice comes from the character. The SDK also reports the agent's current volume as a number from 0 to 1, which the sample uses to make a cube bob while it talks and which could drive a mouth animation. One agent can play several characters: dynamic variables fill placeholders such as {{mood}} in the agent's prompt and first message, so in the sample a yellow happy cube and a red angry cube share a single agent. Overrides go further and replace part of the agent's configuration, such as its first message or voice, for one session, but each override must first be switched on in the agent's Security settings. While a conversation is running, the game can tell the agent what just happened, for example that the player picked up the cellar key, without interrupting it or inserting a line the player never said. Client tools work in the other direction: a tool defined in the agent's settings, such as one called open_door, runs a handler the game registered under the same name, whatever the handler returns goes back to the agent, and a handler can also tell the agent why something failed.

How well this works depends on where the game runs. The SDK chooses the transport, the network connection that carries the conversation, for each build target. The Unity Editor and desktop builds for macOS, Windows and Linux are supported over WebSocket, a persistent two-way connection. WebGL, Unity's target for games that run in a web browser, is supported over WebSocket or WebRTC, the browser standard for real-time audio and video. iOS, Android and XR headsets are in beta. WebGL has the most caveats: the voice plays through the browser's Web Audio API, so some AudioSource settings such as mixer routing and doppler do not carry over, and WebRTC sessions ignore the character's AudioSource entirely, which gives up the voice coming from the character. On iOS and Android the game has to request microphone permission before starting a session.

The most important instruction in the post concerns authentication. For a first test, ElevenLabs tells developers to turn off Authentication in the agent's Security settings so the game can connect with the agent ID alone, and then warns that it must be turned back on before shipping. In the shipped setup, the developer's own server requests a short-lived signed URL, a temporary connection address, and the game uses that instead of the agent ID; the post says never to put an ElevenLabs API key in a game build. A game build is copied onto every player's machine, so anything inside it should be treated as readable by players, which means a shipping game with voice characters needs a server of its own. The SDK is new and installs at version 0.1.0, and the post gives no figures for response latency or cost per conversation, two things that decide whether a talking character feels natural and stays affordable at scale. What it does show is that the connection, positional audio, game-event updates and agent-triggered game actions come packaged, so trying a talking character is quick (ElevenLabs puts its getting started walkthrough at about 10 minutes), while a shipping game still has to supply that server and work around the per-platform caveats.