@Jason @Lons @eisokant @ape Correction: Should actually be closer to 50 tok/sec

Alex Cheema, a co-founder of EXO Labs, had predicted that the upcoming open weights of Qwen 3.8 27B would bring intelligence close to GLM-5.2 to a 36GB MacBook at about 25 tokens per second. In this follow-up he corrects the speed to closer to 50 tokens per second, on the condition that the software uses multi-token prediction, or MTP.

Cheema groups MTP with other speculative decoding methods such as DFlash. A language model normally produces text one token per full pass, so generating many tokens takes many passes in a row. Speculative decoding has a cheaper predictor draft several tokens ahead and lets the full model check them together; the paper that introduced it describes faster sampling without any changes to the outputs. That is why Cheema calls these methods a free lunch with no loss in quality. He adds that the gain depends on the task, because it rises with how often the drafted tokens are accepted, and code, full of predictable tokens like brackets, tends to be accepted more often.

The higher figure depends on software that is not yet standard. Cheema says the MLX ecosystem for running models on Macs has been slow to adopt MTP, and points to MTPLX, a separate project that already supports it; its developer reported more than 72 tokens per second on the earlier Qwen 3.6 27B on a MacBook Pro M5 Max. Cheema says he hopes the Qwen 3.8 27B launch pushes MLX to add MTP support.