Cactus Needle 3: 8-29MB automation models can match DeepSeek V4 Flash
Source
HenryNdubuaku
Author
HenryNdubuaku
Date
Key takeaways · AI-distilled
Needle 3 ships as 8-29MB binaries (25-121M parameters at 2-bit) using an 'Intelligence Laddering' design where every layer from 2 to 20 is itself a deployable subnetwork from one set of weights, rather than training separate model sizes.
Its MLP replaces the standard dense feedforward layer with a 'Monarch Hadamard MLP' — Walsh-Hadamard-initialized Kronecker factor pairs plus per-channel scales and a rank-8 gate — giving roughly O(d√d) compute per token versus the O(d²) a dense 4x-expansion MLP would cost.
On a Mobile Actions benchmark (phone commands scored on exact tool call), the 20-layer Needle 3 scores 86.0 versus LFM2.5 1.2B at 82.4, Qwen3.5 0.8B at 76.0, and Apple's on-device model at 57.6 — rivals evaluated at full f16 against Needle's shipped 2-bit binary.
Every response carries a calibrationHow well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.Full definition → confidence score (the minimum of a judgment on the finished call and its decode probability), letting a deployment act automatically above a chosen threshold, ask for confirmation below it, or escalate to a bigger model.
The team reports that fine-tuningTaking a trained model and training it a bit more on your own examples so it gets better at one specific job.Full definition → just a 4-layer Needle on a narrow task can reach DeepSeek V4 Flash-level performance on that task, though they stress this only holds for narrow, not general-purpose, tasks.
Terms in this piece · Glossary
structured output — Forcing a model's response to match a schema, so downstream code can parse it instead of guessing at prose.
AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.
fine-tuning — Taking a trained model and training it a bit more on your own examples so it gets better at one specific job.
Why it matters
Needle 3 packs tool-calling and JSON structured outputForcing a model's response to match a schema, so downstream code can parse it instead of guessing at prose.Full definition → into 8-29MB binaries that run at thousands of tokens/sec on a Raspberry Pi 5, beating larger on-device models like Apple's on a mobile-action benchmark, a real option for edge AI agentAn AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.Full definition → automation.
