Intel / post

Tencent opens Hy4 preview: 770B sparse MoE with a 1M-token window

Source: x.com
Date: Published Aug 28, 2026

Context

Tencent's Hy4 preview separates total model capacity from the computation used for each piece of text. Its backbone contains 770 billion parameters, but activates 49 billion per token by routing work through selected expert blocks. The model specification also lists a one-million-token , describing how much text the model can work with at once. These numbers describe different properties: the size of the full network, the portion used during generation and the capacity of its working context.

Terms in this piece · Glossary

context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
open weights — A model whose trained parameters are published for anyone to download and run — unlike API-only models you can access but never possess.
mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
calibration — How well a model's confidence matches reality — a calibrated model saying "90% sure" is right about 90% of the time.

Read the source x.com