ZMLX

Why it's ranked here

The case rests on measured, model-specific gains rather than a blanket speed claim. Reported improvements range from neutral to 12.8 percent on an M4 Max, with token-identical greedy outputs. Automatic skipping limits risk, while the validation workflow encourages checking results on the actual machine.

What's good

The project states exactly where acceleration comes from: combining normalization, gating, activation, projection, and recurrent-state work into fewer GPU launches. Its benchmark tables name models, patched module counts, hardware, decoding mode, and neutral outcomes. Users can also auto-benchmark candidate patches before keeping them.

Tradeoffs

This is narrowly tied to macOS 14 or newer, Apple Silicon, and recent MLX releases. Optimized paths apply only to short-sequence decode, while prefill remains untouched. Some models gain little or nothing, full-scale coverage is incomplete, and selected kernels require a custom MLX build.

How to use it well

Use ZMLX when serving or experimenting with supported MLX language models on an Apple Silicon Mac, especially when decode latency matters. Start with automatic matching, then validate output and timing on your model and hardware. Treat the kernel API as an optimization workshop, not a cross-platform inference runtime or a general model-serving solution.

Observed

License

MIT

Primary language

Python

Packaging

Hatchling wheel published as zmlx, installable with pip

Runtime requirements

Python 3.10+, MLX 0.30+, macOS 14+ on Apple Silicon

Interfaces

Python library, command-line tools, and an OpenAI-compatible HTTP server

Optional dependencies

Extras cover MLX-LM integration, development tools, KV cache coding, and kernel discovery

Platform exception

The NumPy-based KVTC subsystem is allowed to import on unsupported hosts