Tencent open-sources Hy4 preview at 770B params and 1M context
- Source
- x.com
- Date
Tencent has released and open-sourced Hy4 preview, a next-generation large language model with 770B total parameters and a 1M+ token context window. Ranked among the top tier of open-source models. Try Hy4 preview. OpenRouter: https://t.co/2g7YeWjkt4 Tencent Cloud TokenHub: https://t.co/hePHnvayx8 WorkBuddy: https://t.co/BTWiHSOGWR CodeBuddy: https://t.co/EOUZ5DL9t5 #TencentAI #Hunyuan
Context
Tencent's Hy4 preview distinguishes total model size from the portion used for each text . Its model card lists 770 billion backbone parameters, with 49 billion activated per token through a design.
The launch announcement also specifies a exceeding one million tokens, the text units available to the model within a request. That capacity makes room for lengthy material, but the capacity figure alone does not establish how accurately the model uses every part of it.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- mixture-of-experts — A model built from many specialist sub-networks where only a few activate per token, giving big-model capability at small-model running cost.
- context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Checking sign-in…
Loading comments…