Vibeleaderboard
Index / tool
Visit voxcpm.com
Category
AI Tools
Rank
Pricing
Open Source
Type
TOOL
Builder
openbmb
Latest release
2.0.3
Date

About

A tokenizer-free text-to-speech system that generates high-quality multilingual speech with voice cloning and creative voice design capabilities. Supports 30 languages and can create new voices from text descriptions alone or clone existing voices from audio samples.

What it does

VoxCPM2 turns text into waveform audio by generating continuous speech representations with a diffusion autoregressive architecture. It offers plain synthesis, described voice creation, reference-based timbre matching, transcript-guided audio continuation, and chunked streaming output.

Why it's ranked here

The project covers unusually broad speech workflows through one model and exposes them through Python, a command line tool, and a browser demo. Its strongest case is practical control: users can steer expression, preserve a reference timbre, or prioritize detailed continuation from transcribed audio.

What's good

It outputs 48 kHz audio directly, accepts 16 kHz reference clips, and includes built-in super-resolution. Generation supports fixed seeds, optional reference denoising, text normalization, streaming chunks, batch processing, and optional word or character timestamps. The same package also includes LoRA training support.

Tradeoffs

The model has two billion parameters, while the documented quick-start stack requires recent Python, PyTorch, and CUDA versions. CPU and Apple MPS are supported in device selection, but optimization only runs on CUDA. MPS defaults to float32 because lower precision can destabilize generation. Transcript-guided cloning cannot be combined with style control.

How to use it well

Use it for local or hosted speech production where multilingual output, designed voices, or controlled cloning matter. Start with the browser demo, then automate repeatable jobs through Python or the command line. Use fixed seeds for reproducibility and transcript-guided continuation when fidelity matters most. It does not provide speech recognition as its primary product.

Technical notes+

pyproject.toml defines a setuptools package under src, a voxcpm console entry point, Python 3.10 or newer, and optional timestamp dependencies. src/voxcpm/core.py selects the model architecture from checkpoint configuration, downloads Hub snapshots, validates prompt and reference inputs, and returns arrays or streaming generators. src/voxcpm/model/voxcpm2.py combines MiniCPM language components, local encoding, diffusion generation, reference-audio tokens, LoRA injection, and AudioVAE V2 decoding. src/voxcpm/model/utils.py resolves CUDA, MPS, or CPU and forces float32 on MPS by default. app.py initializes the synthesis and recognition models only when first requested.

Observed

License
Apache-2.0 for code and weights
Primary language
Python
Packaging
Setuptools package installable as voxcpm with pip
Interfaces
Python library, command line tool, and Gradio web demo
Runtime devices
CUDA, Apple MPS, and CPU
Python support
Python 3.10 or newer; README limits the quick start to versions below 3.13
Optional surface
Timestamp alignment is available through the timestamps package extra

Read from README.md, pyproject.toml, app.py, src/voxcpm/cli.py, src/voxcpm/core.py, src/voxcpm/__init__.py, src/voxcpm/zipenhancer.py, src/voxcpm/model/utils.py, src/voxcpm/model/voxcpm.py, src/voxcpm/model/voxcpm2.py, src/voxcpm/training/data.py.

What it can do

  • Generate speech from text in multiple languages

    Text and target language selection from 30 supported languages48kHz studio-quality audio speech

  • Clone existing voice from audio sample

    Audio sample of target voice and text to be spokenSpeech audio in the cloned voice

  • Create new voice from text description

    Text description of desired voice characteristicsSynthesized speech with voice matching the description

  • Convert text to speech without tokenization preprocessing

    Raw text in any of 30 supported languagesDirect audio generation without intermediate token processing

  • Generate controllable voice variations

    Base voice sample or description with control parametersModified voice output with adjusted characteristics

Tags

ttstext-to-speechvoice-cloningmultilingualspeech-synthesisaudiovoice-designai

Tech Stack

Python

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.