MLX Audio
github.com/blaizzy/mlx-audio- Category
- AI Tools
- Rank
- No. 744Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- blaizzy
- GitHub
- 7.7k stars
- Latest release
- v0.4.8
- Added
- May 22, 2026
About
Speech library built on Apple's MLX — text-to-speech, speech-to-text, and speech-to-speech with efficient on-device inference on Apple Silicon.
What it does
MLX Audio gives Mac developers one toolkit for running many speech model families locally. It loads checkpoints from Hugging Face or local storage, then exposes generation, transcription, cloning, separation, enhancement, diarization, and turn detection through Python, command-line tools, a web interface, and an OpenAI-compatible server.
Why it's ranked here
The unusually broad model coverage makes this more than a single-model wrapper. Streaming works across Python, command-line, HTTP, and real-time WebSocket workflows. Quantization and ready-made low-precision checkpoints address memory and speed. The catch is fundamental: the documented runtime target is an Apple Silicon Mac.
What's good
The interfaces cover both experimentation and integration. Developers can stream audio chunks before synthesis finishes, return incremental transcription as newline-delimited JSON, or process live microphone input over WebSockets. Voice cloning spans several model families, while conversion supports multiple bit widths and mixed-precision recipes. Optional dependency groups keep speech, server, documentation, and development tooling separable.
Tradeoffs
Apple Silicon is a documented requirement, so this is not a general inference layer for Linux, Windows, or non-Apple accelerators. Model behavior varies: some cloning workflows need matching reference transcripts, and aggressive quantization can reduce speech quality. Model weights remain resident during streaming, so streaming changes delivery latency rather than peak memory use. Non-WAV encoding also requires ffmpeg.
How to use it well
Use it for Mac-based speech prototyping, local batch jobs, low-latency playback, transcription services, or applications already speaking OpenAI-style audio APIs. Start with a pre-quantized checkpoint, test output by listening, then tune chunk intervals and precision. It does not cover cross-platform deployment; use another runtime when Apple Silicon cannot be assumed.
Technical notes+
pyproject.toml defines a setuptools Python package requiring Python 3.10+, core MLX and audio dependencies, optional stt, tts, sts, server, dev, and docs groups, plus five CLI entry points. docs/guides/streaming.md documents iterator-based TTS chunks, streaming STT, HTTP streaming, and /v1/audio/transcriptions/realtime WebSockets. docs/guides/quantization.md covers affine, MXFP4, MXFP8, NVFP4, and mixed-precision conversion through mlx_audio.convert. docs/api-reference/audio-io.md assigns WAV and MP3 decoding to miniaudio, with ffmpeg used for several other codecs and non-WAV encoding. Tests are organized inside mlx_audio/tts/tests/, mlx_audio/stt/tests/, and mlx_audio/sts/tests/ according to docs/contributing/dev-setup.md.
Observed
- License
- MIT
- Primary language
- Python
- Installation
- Published as mlx-audio for pip and uv; editable installs support optional feature groups.
- Interfaces
- Python library, command-line tools, web interface, OpenAI-compatible REST API, and real-time transcription WebSocket.
- Platform support
- Documented for Apple Silicon Macs with M1, M2, M3, or M4 chips.
- Python requirement
- Python 3.10 or newer.
- Packaging
- Setuptools build backend with separate extras for TTS, STT, STS, server, development, and documentation.
- Companion platform package
- A separate Swift package provides on-device TTS integration for macOS and iOS.
Read from README.md, pyproject.toml, docs/index.md, docs/guides/index.md, docs/models/index.md, docs/guides/streaming.md, docs/api-reference/stt.md, docs/api-reference/tts.md, docs/contributing/index.md, docs/api-reference/index.md, docs/guides/quantization.md, docs/guides/voice-cloning.md, docs/getting-started/index.md, docs/api-reference/audio-io.md, docs/contributing/dev-setup.md.
What it can do
Convert text to speech
Text content → Audio speech file
Convert speech to text
Audio speech file → Transcribed text
Convert speech to speech
Audio speech file → Modified audio speech file
Perform on-device speech processing
Audio or text data → Processed speech results without cloud dependency
Run efficient inference on Apple Silicon
Speech or text data → Optimized processing results using Apple's MLX framework
Tags
Tech Stack
Comments (0)
No comments yet
Indexed by a proprietary survey. Corrections welcome.