Intel / article

Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

Source: prismml.com
Author: JonSchneider
Date: Published Sep 17, 2026

From the Daily Brief

PrismML's Bonsai 2 uses ternary weights, effectively 1.76 bits per parameter, to compress a 27-billion-parameter multimodal model into a 5.9GB file, a roughly ninefold reduction from full precision. Despite that compression, the model retains 98.2 percent of the benchmark performance of the full-precision Qwen3.8 27B it is derived from, across coding, reasoning, and agentic tool use, and keeps a 262K-token context window. It ships under Apache 2.0.

The result matters because 27B-class models have generally been too large to run locally on consumer hardware without accuracy loss that made them impractical. A model this capable fitting under 6GB puts it within reach of a single consumer GPU or a high-end laptop, without the usual tradeoff of gutting reasoning or agentic performance to hit that size. For teams that need to run inference locally for cost, latency, or data residency reasons, Bonsai 2 is a concrete option rather than a compromise.

Read the 2026-09-18 Brief →

Context

PrismML reports that Bonsai 2 27B compresses a model based on Qwen3.8 27B into a 5.9 GB footprint, which it describes as more than nine times smaller than the full-precision model, by representing weights with three values plus scaling information. Its release report compares an aggregate score of 83.9 with the full-precision model's 85.4, producing the advertised 98.2% retention. That percentage describes the relationship between two aggregate scores, not the fraction of real-world requests answered correctly. The result supports a narrower deployment proposition: substantially less model storage while retaining much of the measured capability on the company's benchmark suite.

Terms in this piece · Glossary

benchmark — A standard public test set for comparing AI models — the shared scoreboards behind every "model X beats model Y" claim.
multimodal — A model that works with more than text — reading images, audio, or video, and sometimes generating them too.
tool use — A model's ability to call external functions — run code, search the web, edit files — instead of only generating text.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.

Read the source prismml.com