Ling-3.1-flash open-sourced: 560B MoE, 1M context, 4-GPU serving

Ant Group's Ling team, publishing as inclusionAI, has released open weights for Ling-3.1-flash under the MIT license. The model was announced two weeks earlier with a promise to open-source it soon. Its model card describes about 560 billion parameters in total, about 25 billion active for each token, and a context window of up to 1 million tokens. It is a mixture-of-experts model: its expert layers hold 512 small expert networks, and a router sends each token through 8 of them plus 1 shared expert. Each token therefore touches only a small slice of the network, but the whole network still has to sit in GPU memory to serve it, which is why the headline size matters for hardware.

The four-GPU claim in the title is about shrinking that memory footprint. In the release thread, Ant Ling says storing the expert weights in MXFP4, a 4-bit number format, and the dense layers in block-FP8, an 8-bit one, cuts the weights from about 534GB to about 292GB, and that a tested setup on Nvidia Hopper GPUs went from 8 GPUs to 4. It also gives two performance figures: 8 GPUs averaged 604 milliseconds to the first token, and 4 GPUs reached about 1.36 queries per second. Ant Ling notes that these came from separate tests with different workloads, so they are not a like-for-like pair. The model card's own quickstart recipe uses 8 H200 GPUs, or 4 of Nvidia's newer GB300s, and does not itself describe the compressed build, so the four-Hopper result rests on the thread.

The rest of the thread positions the model for frequent business use. Ant Ling cites a score of 41 on the Artificial Analysis Intelligence Index, which Artificial Analysis reported as up from 20 for Ling 3.0 Flash, along with about 210 output tokens per second and a blended price of about $0.19 per million tokens on that benchmark's measure, noting that provider prices may differ. For generated front-end work, it says the model ranks third among open-weight models on Design Arena for React Native screens and 3D, fourth for SVG and fifth for websites. The SGLang and vLLM serving projects support the model.