PrismML ships Ternary Bonsai 2 small models for on-device agents

PrismML's new Ternary Bonsai 2 models are compressed versions of Qwen's Qwen3.5 2B, 4B and 9B in which every language-model weight is stored as one of three values, −1, 0 or +1 (PrismML). That works out to about 1.7 bits per weight instead of the usual 16, applied across the whole language model rather than leaving some layers at high precision, according to PrismML's documentation (docs). The result is files of 0.43GB, 0.95GB and 2GB, which PrismML says is more than nine times smaller than the originals. Size is not the only gain: generating text is limited mainly by how fast weights can be read from memory, so smaller weights also mean faster output, and PrismML reports up to five times the throughput and three times the energy efficiency of the full-precision models on an RTX 4090 and an M5 Pro, and up to 34 tokens per second for the 9B on an iPhone 18 Pro Max. The three join Bonsai 2 27B, released last month with the same approach.

PrismML's own comparison against the uncompressed Qwen3.5 models, across 19 benchmarks, shows where the compression costs little and where it costs more. On math, coding and instruction following the 9B stays essentially level with its original, and the 2B even scores higher on some instruction-following tests, such as 48.00 versus 34.33 on IFBench; PrismML says this release focused on following instructions and avoiding unnecessary output. Knowledge-heavy and agentic tasks lose more: on GPQA Diamond the 9B scores 69.9 against 82.53 for full-precision Qwen3.5 9B, and on the Tau2Bench agent benchmark the 4B keeps 75.8% of its original score. In the knowledge-and-reasoning category the loss grows as the models shrink, from 93.0% retained at 9B to 82.9% at 2B. These are PrismML's figures; the whitepaper describing its compression and evaluation methods could not be read for this summary, and the demonstration videos were not transcribed.

Running the models takes more than a file that matches the right format. The weights are stored in a rotated form, so the software running them must apply a matching transform to its intermediate values, and PrismML's docs warn that an older llama.cpp build will load the file and produce gibberish rather than refusing it (docs). Upstream llama.cpp now runs one of the three file layouts PrismML publishes, the mlx-vlm library loads the Apple versions natively, and the weights are released under Apache 2.0.