Whistle: Speech to Text in 16.9 MB

Whistle is a speech-to-text model from Cactus Compute that is 16.9 MB and runs on a CPU. In the Cactus post, measured on an Apple M4 Pro with 10 seconds of audio, Whistle takes 11.1 ms to produce its first token and then decodes 1,319 tokens per second (a token here is a short piece of text the model emits while writing the transcript). Whisper base, at 145.3 MB, is listed at 73.2 ms and 266 tokens per second, and Moonshine tiny v2, at 41.9 MB, at 22.8 ms and 262.

Accuracy is mixed. The post says Whistle has lower word error rate on LibriSpeech, SPGISpeech, Earnings-22 and the FLEURS average, while Whisper base is ahead on TED-LIUM, AMI and the MLS average. Cactus measured Whistle's error rates and the speeds itself; the other two models' error rates are their authors' published figures, so this is a vendor comparison.

Clips are limited to 30 seconds and seven languages, and speed was reported on one machine, so the result shows a very small CPU-only transcriber can be fast and competitive on some test sets, not that it replaces larger speech models.