Qwen3.8-Flash-Next open-sources a Qwen4 architecture preview at 6B active

Qwen released two related things here. Qwen3.8-Flash-Next is a set of open weights that its model card calls an experimental preview of the architecture that will underpin Qwen4. Qwen3.8-Flash is the production version built on it and sold through Qwen Cloud, with extras the open release lacks, such as a 1M-token context by default and official built-in tools. The open model's own native context is 262,144 tokens, which the card says can be stretched to about 1M with a RoPE scaling method called YaRN. The headline of 6B active parameters needs unpacking. The card lists a 125B-parameter mixture-of-experts language model with 6B activated, plus 51B of n-gram embeddings and 4B for multi-token prediction. Those add up to the 180B model size shown on the Hugging Face page, so the download is much larger than the per-token compute figure suggests.

The n-gram embedding is the unusual part. Besides the normal table that gives each token a learned vector, the model keeps a separate table of 20 million entries indexed by short bigrams and trigrams and used at layer 2. The card argues that embeddings offer a way to add parameters with less computation than adding experts, and that they are easier to offload, which suits accelerators short on memory. In a follow-up analysis, SemiAnalysis drew the practical consequence: the table could sit in slower, cheaper tiers of DRAM. The other architectural changes are Qwen Sparse Attention, which picks context in micro-blocks rather than token by token to cut long-context latency, and a Gated Residual design that controls how information flows between layers.

The benchmark results are Qwen's own, and the footnotes matter for comparisons. For DeepSWE 1.1, the 58.7 is the better of two agent harnesses, and the card notes the model did best on mini-SWE-agent. For SWE-bench Pro, Qwen ran every model in the Claude Code harness on a version of the benchmark with problematic tasks corrected, except Claude Opus 4.6, whose score is the officially published one. CoWorkBench, where the model scores 73.9, is an in-house benchmark. The announcement also says the model was trained at one-ninth the cost of Qwen3.7-Plus and quotes API prices of $0.16 per million input tokens and $0.47 per million output tokens. The cost figure can't be checked against the technical report, which wasn't readable, and Qwen's later API post lists input at $0.15.