Open-LLM-VTuber
github.com/open-llm-vtuber/open-llm-vtuber- Category
- AI Agents
- Rank
- No. 835Tools index
Previous survey · No. 815 ·
- Type
- APP
- Builder
- open-llm-vtuber
- GitHub
- 13.8k stars
- Latest release
- v1.2.1
- Date
About
Hands-free voice conversation with any LLM, with interruption and Live2D avatars.
What it does
Open-LLM-VTuber assembles speech recognition, language models, speech synthesis, translation, visual input, and an animated character into one configurable companion. It runs through web or desktop clients, can use local models or cloud services, and stores chat logs for later continuation.
Why it's ranked here
The breadth is unusually practical: multiple model, speech, and avatar backends sit behind a shared configuration system, while local execution keeps private conversations on the user's machine. The project is still early, however, and the announced rewrite makes the current generation a less stable foundation for long-term extension.
What's good
It supports Windows, macOS, and Linux, including CPU, NVIDIA, non-NVIDIA, and some macOS acceleration options. Camera and screen perception, expression mapping, proactive speech, translation before synthesis, persistent chat logs, and headphone-free interruption make the character more than a decorated chatbot. MCP support also lets configured agents use external tools.
Tradeoffs
Long-term memory is currently absent, despite persistent conversation logs. Remote microphone access requires HTTPS or localhost, so phone or second-machine use needs a reverse proxy and certificate setup. The maintainers describe the project as early-stage, are planning a complete rewrite, and ask contributors not to submit new feature requests against the current generation.
How to use it well
Pick it for a local AI character, desktop pet, VTuber experiment, or voice interface where swapping model and speech providers matters. Start with supported modules and configuration, then customize the persona, voice, expressions, and Live2D model. Do not treat chat-log persistence as durable long-term memory, or expect the current architecture to remain unchanged through the planned rewrite.
Technical notes+
pyproject.toml defines a Python 3.10 through 3.12 project with FastAPI, Uvicorn, ONNX Runtime, Torch, MCP, and numerous hosted-model and speech SDKs; requirements.txt is exported by uv. src/open_llm_vtuber/server.py builds the FastAPI application, enables wildcard CORS, mounts frontend and model assets, and conditionally registers a WebSocket proxy. src/open_llm_vtuber/routes.py exposes /client-ws, /proxy-ws, /asr, and /tts-ws; the ASR route assumes a 44-byte WAV header and signed 16-bit PCM, while TTS sentence streaming splits on periods. src/open_llm_vtuber/service_context.py constructs configurable ASR, TTS, VAD, translation, agent, Live2D, and MCP components per session. src/open_llm_vtuber/websocket_handler.py routes typed client messages and shares configured engines across cloned session contexts. Its disconnect path removes the session context before attempting to retrieve and close it, so that cleanup branch cannot reach the removed context.
Observed
- License
- MIT for the project code; bundled Live2D sample models use separate Live2D licenses and may require additional commercial permissions.
- Primary language
- Python, packaged as open-llm-vtuber.
- Python support
- Requires Python 3.10 or newer and earlier than Python 3.13.
- Install surface
- uv-managed Python project with an exported requirements file; the README also links a Docker image and external quick-start documentation.
- Interfaces
- Web client, desktop client, FastAPI HTTP endpoints, WebSocket endpoints, and optional MCP tool integration.
- Platform support
- Windows, macOS, and Linux, with local CPU or GPU execution and cloud API options.
Read from README.md, pyproject.toml, requirements.txt, src/open_llm_vtuber/server.py, src/open_llm_vtuber/routes.py, src/open_llm_vtuber/chat_group.py, src/open_llm_vtuber/live2d_model.py, src/open_llm_vtuber/proxy_handler.py, src/open_llm_vtuber/message_handler.py, src/open_llm_vtuber/service_context.py, src/open_llm_vtuber/websocket_handler.py.
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.