Ling-3.0-flash gets INT4 and MXFP4 builds for a single DGX Spark
- Source
- x.com
- Date
🚀 Today, we’re releasing INT4 and FP4 (MXFP4) variants of Ling-3.0-flash. Both run end to end on a single NVIDIA DGX Spark via our Spark-adapted SGLang path. For FP4, W4A16 is the stable default, while W4A8 is tuned for higher throughput. The efficiency and accuracy of the quantized models are still evolving. We will continue updating the inference implementation and quantized weights over time.
Context
A day after publishing BF16 and FP8 weights for Ling-3.0-flash, Ant Ling released INT4 and MXFP4 versions that store most of the model's weights in 4 bits instead of 16. Ling-3.0-flash is a model with 124 billion total parameters, according to its 4-bit model card, which says the 4-bit is applied group by group to the routed expert weights, the specialist sub-networks the model switches between. At 16 bits, 124 billion parameters take roughly 250 GB, so shrinking most of them to a quarter of that size is what lets Ant say both builds run end to end on a single NVIDIA DGX Spark, a desktop AI computer, through a version of the SGLang serving engine that Ant adapted for that machine. INT4 stores each weight as a plain 4-bit integer. MXFP4 is a 4-bit floating-point format in which small groups of values share a common scale factor.
The W and A in Ant's naming refer to weights and activations, the intermediate values the model computes as it runs. For the FP4 build, Ant says W4A16, which keeps 4-bit weights but computes with 16-bit activations, is the stable default, while W4A8, which also cuts activations to 8 bits, is tuned for higher throughput. The model card puts a number on the accuracy cost, without saying which configuration it measured: against BF16, GPQA-diamond falls from 84.97 to 83.65 for INT4 and 83.68 for FP4, IFBench from 74.49 to 72.20 and 73.87, SciCode from 41.24 to 39.35 and 38.43, and ArcPrize from 68.75 to 67.56 and 68.06. Ant says the efficiency and accuracy of the quantized models are still evolving and that it will keep updating both the weights and the code, so these are early figures from the company's own tests on four benchmarks.
- mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
- quantization — Shrinking a model by storing its numbers less precisely — like rounding — so it runs faster and fits on smaller hardware, at a small quality cost.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
Checking sign-in…
Loading comments…






