We have open-sourced HY-1.8B-2Bit, a high-efficiency 2-bit LLM built for on-device deployment. This model scales the 1.8B base down to an effective 0.3B parameter footprint, requiring only 600MB of storage, making it smaller than many mobile apps.
🔹 Ultra-Low-Bit Strategy: Uses QAT (Quantization-Aware Training) to reach a 2-bit representation (0.3B bit-equivalent size).
🔹 Dual-CoT Reasoning: Retains sophisticated Dual Chain-of-Thought capabilities despite radical precision reduction.
🔹 Performance: 3-8x faster prefill on Apple M4 and MediaTek Dimensity 9500; 2-3x faster token generation on-device.
🔹 Benchmark Gains: Achieves a 17% average accuracy lead over models of equivalent size.
🔹 Hardware Synergy: Optimized for Arm SME2 and modern consumer silicon.
HY-1.8B-2Bit is available now in GGUF format for seamless integration into edge-based inference engines.
Project Page: https://t.co/pFp6vgpooa
Weights: https://t.co/Pvua0eec3L
GGUF Version: https://t.co/wdRIsUqwFb
Technical Report:
quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
chain-of-thought — Having a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Why it matters
QAT to 2 bits with chain-of-thoughtHaving a model write out intermediate reasoning steps before its answer, which markedly improves performance on hard problems.Full definition → reasoning intact means a 600MB model that fits inside a mobile app bundle, and the GGUF release drops straight into existing edge inferenceRunning a trained model to get answers — the phase where AI is actually used, as opposed to trained.Full definition → engines.
