OpenAI Realtime Voice Component
https://github.com/openai/realtime-voice-component- Category
- AI Tools
- Rank
- No. 645Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- openai
- GitHub
- 887 stars
- Date
About
React component for the OpenAI Realtime API. Drop in voice-to-voice chat with the right audio handling, interruption, and event wiring.
What it does
It turns spoken requests into tightly scoped actions inside React browser interfaces. Developers describe allowed actions with Zod schemas, while a controller manages WebRTC sessions, transcripts, tool execution, connection state, and optional visual confirmation.
Why it's ranked here
This is a strong reference for constrained voice control because it keeps application code responsible for state, validation, and success. Its practical demos and explicit architecture guidance offset an experimental distribution model and intentionally narrow scope.
What's good
The design separates session control, React binding, launcher UI, and cursor feedback. Tools validate arguments through Zod before reaching application handlers. It supports shared controllers across routes, several output modes, configurable authentication, tool history, event callbacks, and reduced-motion cursor behavior.
Tradeoffs
The package is experimental, private, and unavailable from npm, so adoption requires a local checkout. The launcher is not a transcript interface or built-in push-to-talk surface. Non-React runtimes, custom audio handling, broad orchestration, hosted tools, and server-side agent systems need another layer.
How to use it well
Use it for React applications where voice should trigger a small set of visible, app-owned actions. Begin with one tool, keep validation in existing handlers, proxy session setup through your backend, and synchronize fresh screen state. Choose another SDK for general orchestration or non-browser systems.
Technical notes+
src/index.ts marks the entry as client-side and exports the controller, React hook, launcher, ghost cursor helpers, tool builder, and public types. src/defineVoiceTool.ts requires Zod-like schemas, while src/schema.ts converts them to JSON Schema and parses arguments. src/useVoiceControl.ts binds controller snapshots through useSyncExternalStore and destroys internally owned controllers. src/voiceControlController.ts handles WebRTC transport, session configuration, transcript events, tool-call records, authentication paths, and normalized errors. package.json builds ESM, CommonJS, declarations, and CSS with tsup, but remains private and targets local installation.
Observed
- License
- Apache-2.0
- Primary language
- TypeScript with React and TSX interfaces
- Distribution
- Private package intended for installation from a local Git checkout, not npm
- Interface
- React browser library with a controller, hook, launcher widget, and optional ghost cursor
- Transport
- Browser WebRTC with a server-proxied session endpoint as the canonical authentication path
- Packaging
- Build outputs include ESM, CommonJS, TypeScript declarations, and CSS
- Runtime requirements
- Node.js 20 or newer for the package tooling
- Peer dependencies
- React 18.3 or 19, React DOM 18.3 or 19, and Zod 4.3.6
Read from README.md, package.json, src/index.ts, src/types.ts, src/schema.ts, src/useGhostCursor.ts, src/defineVoiceTool.ts, src/useVoiceControl.ts, src/voiceControlController.ts, docs/README.md, docs/api-reference.md, docs/authentication.md, docs/getting-started.md, docs/demo-architecture.md, docs/widget-and-cursor.md.
What it can do
Enable voice-to-voice conversations
User's spoken audio → AI-generated spoken responses
Handle audio input processing
Raw microphone audio stream → Processed audio data for API transmission
Manage conversation interruptions
User speech during AI response → Interrupted AI speech and new response generation
Wire API event handling
OpenAI Realtime API events → Appropriate component state updates and audio playback
Integrate voice chat into React applications
React component props and configuration → Functional voice chat interface
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.