
VoxCPM2
github.com/openbmb/voxcpm- Category
- AI Tools
- Rank
- No. 206Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- openbmb
- GitHub
- 36.9k stars
- Latest release
- 2.0.3
- Date
About
A tokenizer-free text-to-speech system that generates high-quality multilingual speech with voice cloning and creative voice design capabilities. Supports 30 languages and can create new voices from text descriptions alone or clone existing voices from audio samples.
What it does
VoxCPM2 turns text into waveform audio by generating continuous speech representations with a diffusion autoregressive architecture. It offers plain synthesis, described voice creation, reference-based timbre matching, transcript-guided audio continuation, and chunked streaming output.
Why it's ranked here
The project covers unusually broad speech workflows through one model and exposes them through Python, a command line tool, and a browser demo. Its strongest case is practical control: users can steer expression, preserve a reference timbre, or prioritize detailed continuation from transcribed audio.
What's good
It outputs 48 kHz audio directly, accepts 16 kHz reference clips, and includes built-in super-resolution. Generation supports fixed seeds, optional reference denoising, text normalization, streaming chunks, batch processing, and optional word or character timestamps. The same package also includes LoRA training support.
Tradeoffs
The model has two billion parameters, while the documented quick-start stack requires recent Python, PyTorch, and CUDA versions. CPU and Apple MPS are supported in device selection, but optimization only runs on CUDA. MPS defaults to float32 because lower precision can destabilize generation. Transcript-guided cloning cannot be combined with style control.
How to use it well
Use it for local or hosted speech production where multilingual output, designed voices, or controlled cloning matter. Start with the browser demo, then automate repeatable jobs through Python or the command line. Use fixed seeds for reproducibility and transcript-guided continuation when fidelity matters most. It does not provide speech recognition as its primary product.
Technical notes+
pyproject.toml defines a setuptools package under src, a voxcpm console entry point, Python 3.10 or newer, and optional timestamp dependencies. src/voxcpm/core.py selects the model architecture from checkpoint configuration, downloads Hub snapshots, validates prompt and reference inputs, and returns arrays or streaming generators. src/voxcpm/model/voxcpm2.py combines MiniCPM language components, local encoding, diffusion generation, reference-audio tokens, LoRA injection, and AudioVAE V2 decoding. src/voxcpm/model/utils.py resolves CUDA, MPS, or CPU and forces float32 on MPS by default. app.py initializes the synthesis and recognition models only when first requested.
Observed
- License
- Apache-2.0 for code and weights
- Primary language
- Python
- Packaging
- Setuptools package installable as voxcpm with pip
- Interfaces
- Python library, command line tool, and Gradio web demo
- Runtime devices
- CUDA, Apple MPS, and CPU
- Python support
- Python 3.10 or newer; README limits the quick start to versions below 3.13
- Optional surface
- Timestamp alignment is available through the timestamps package extra
Read from README.md, pyproject.toml, app.py, src/voxcpm/cli.py, src/voxcpm/core.py, src/voxcpm/__init__.py, src/voxcpm/zipenhancer.py, src/voxcpm/model/utils.py, src/voxcpm/model/voxcpm.py, src/voxcpm/model/voxcpm2.py, src/voxcpm/training/data.py.
What it can do
Generate speech from text in multiple languages
Text and target language selection from 30 supported languages → 48kHz studio-quality audio speech
Clone existing voice from audio sample
Audio sample of target voice and text to be spoken → Speech audio in the cloned voice
Create new voice from text description
Text description of desired voice characteristics → Synthesized speech with voice matching the description
Convert text to speech without tokenization preprocessing
Raw text in any of 30 supported languages → Direct audio generation without intermediate token processing
Generate controllable voice variations
Base voice sample or description with control parameters → Modified voice output with adjusted characteristics
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.