Liquid AI Ships Speculative Decoding for Its Vision-Language Models

• Liquid AI's experimental DSpark drafter brings speculative decoding to LFM2.5-VL-3B: a lightweight model proposes several tokens ahead and the target model verifies them in one pass, which Liquid says leaves output quality unchanged.
• Reported speedups: up to 3.13x faster decoding (2.62x end to end) with MLX on M5 Max, 2.14x (1.77x) with llama.cpp on M3 Ultra, and 2.66x (2.27x) with SGLang on H100.
• The figures cover six vision-language task categories at batch size 1 and temperature 0, measured with Liquid's Pipette benchmarking tool; the post does not report results for batched or sampled decoding.

• speculative decoding — A speed trick where a small model drafts several tokens ahead and the big model verifies them in one pass, often doubling generation speed.
• LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
• token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.

Provides real cross-platform speedup numbers for speculative decoding applied to vision-language models, useful for anyone deploying LFM2.5-VL locally or at scale.

• postDSpark draft models cut LFM2.5 agent latency roughly in half
• postTwo 350M retrieval models: one dense, one late interaction
• postPipette benchmarks local models as model, quant, runtime and device
• postLiquid AI's 230M model runs agentic tool calls on a Raspberry Pi

• articleAccelerating vision-language models with LFM2.5-VL-DSparkhuggingface.co
• articleLFM2.5 VL 3B DSpark releasedLiquid AI model releases
• articleUp to 3.2x Faster Inference with LFM2.5-DSparkhuggingface.co

Checking sign-in…

Loading comments…