OpenAI Realtime Agents
https://github.com/openai/openai-realtime-agents- Category
- AI Agents
- Rank
- No. 346Tools index
- Pricing
- Open Source
- Type
- AGENT
- Builder
- openai
- GitHub
- 7.0k stars
- Date
About
Reference patterns for building voice agents on the OpenAI Realtime API — handoffs, tool use, and low-latency speech-to-speech.
What it does
A runnable Next.js demonstration for testing conversational agent architectures in the browser. One pattern keeps routine dialogue in a realtime model and delegates harder work to a text supervisor. Another moves callers among specialist agents through explicit handoff graphs. The interface includes transcripts, event logs, text input, push-to-talk, codec selection, and audio recording.
Why it's ranked here
The project earns attention because it turns architectural choices into working, inspectable scenarios. The supervisor design offers gradual migration from an existing text agent, while sequential handoffs keep specialist instructions and tools separated. Its value is strongest as a prototype and teaching aid, not as a finished product foundation.
What's good
The examples go beyond toy conversation. The retail flow covers authentication, returns, sales, escalation, state-machine prompting, moderation, and specialist transfers. Developers can inspect client and server events beside the transcript, switch scenarios, test push-to-talk behavior, compare wide-band and phone-line codecs, and export combined conversation audio as WAV.
Tradeoffs
This is a private demo package, not a published library. Running it requires an OpenAI API key and local Next.js setup. Supervisor delegation adds latency, with the documented example showing roughly two seconds before the deferred answer begins. It can also produce repetitive holding phrases. Codec selection relies on a global workaround because the SDK lacks direct support.
How to use it well
Use it when prototyping browser-based voice support flows or deciding between supervisor delegation and specialist handoffs. Start with an existing text agent, move selected tasks into realtime dialogue, and validate each boundary through transcripts and event logs. It does not replace the underlying Agents SDK, supply production deployment infrastructure, or provide a complete human escalation service.
Technical notes+
The private npm app in package.json uses Next.js 15, React 19, TypeScript, @openai/agents, OpenAI, Zod, and Tailwind CSS. src/app/App.tsx obtains an ephemeral key from /api/session, connects scenario-specific RealtimeAgent arrays, applies an output moderation guardrail, sends Realtime events, controls server VAD for push-to-talk, and reconnects when the root agent changes. src/app/agentConfigs/index.ts registers simpleHandoff, customerServiceRetail, and chatSupervisor, with the latter as default. src/app/components/Events.tsx and src/app/components/Transcript.tsx expose event payloads, breadcrumbs, messages, guardrail results, transcript copying, and audio export. src/app/hooks/useAudioDownload.ts merges microphone and remote streams with Web Audio, records WebM, then uses src/app/lib/audioUtils.ts to produce mono 16-bit PCM WAV. src/app/lib/codecUtils.ts sets audio codec preferences on WebRTC transceivers.
Observed
- Primary language
- TypeScript, implemented as a Next.js application with React components.
- Packaging
- Private npm package named realtime-examples.
- Install and run surface
- Install dependencies with npm, provide an OpenAI API key, and start the Next.js development server.
- Interface
- Browser application using the OpenAI Realtime API and OpenAI Agents SDK.
- Included scenarios
- Chat supervisor, sequential handoff, and retail customer-service configurations are registered.
- Browser capabilities
- Supports transcripts, event inspection, text messages, push-to-talk, audio playback, codec selection, and WAV export.
Read from README.md, package.json, src/app/App.tsx, src/app/page.tsx, src/app/types.ts, src/app/layout.tsx, src/app/lib/envSetup.ts, src/app/lib/audioUtils.ts, src/app/lib/codecUtils.ts, src/app/agentConfigs/index.ts, src/app/agentConfigs/types.ts, src/app/components/Events.tsx, src/app/components/Transcript.tsx, src/app/contexts/EventContext.tsx, src/app/hooks/useAudioDownload.ts.
What it can do
Convert speech to speech with low latency
Voice audio input → Voice audio response
Hand off conversations between different agents
Conversation context and handoff trigger → Transferred conversation to appropriate agent
Execute tools and functions during voice conversations
Voice commands requesting tool usage → Tool execution results delivered via speech
Process real-time voice interactions
Streaming audio input → Real-time conversational responses
Build voice-enabled AI agents
Agent configuration and voice interaction patterns → Deployed voice agent application
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.