Intel / article

Integrating Complex External Agents

Source: elevenlabs.io
Author: ElevenLabs editorial sitemap
Date: Published Oct 9, 2026

Context

Adding voice to a complex requires connecting each spoken request to the right ongoing task. ElevenLabs' integration design passes a shared session identifier through its voice service to a developer-built intermediary. That intermediary matches requests to the external agent and can group several conversations into one customer interaction. The external agent retains the substantive task information and business logic; the intermediary routes messages. This division lets a voice call participate in work already underway through other channels.

Key takeaways · AI-distilled

ElevenLabs Agents can sit in front of an external orchestrator via its Custom option, provided the orchestrator exposes OpenAI's Chat Completions or Responses API format, which frameworks like LangChain, LangGraph and CrewAI support.
The post splits agent state into internal state (conversation flow, interruptions, active speaker, detected intents, reasoning trace, active goals) and external state (other users and systems, tools and knowledge bases, and ongoing dependent tasks).
The core integration pattern is proxying one unique session identifier across all services, passed as an LLM parameter in the request extra body, which lets the voice layer and the external agent read and update the same state.

Terms in this piece · Glossary

AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.

Read the source elevenlabs.io