Tencent Opens Hy4 Preview: 770B MoE Weights, 1M Context

The largest open-weight release of the week comes from Tencent, and it is a clear step up in scale rather than a marginal refresh. Hy4 Preview, published on August 28 under the Apache 2.0 license, carries 770 billion total parameters with 49 billion active per token, a context window of more than 1 million tokens, and a 1.56 TB footprint on Hugging Face. July's Hy3 had 295 billion total, 21 billion active and a 256K window. In three months the company has roughly doubled and a half the model and quadrupled the context.

What is inside the model

Hy4 Preview is a Mixture-of-Experts model with 78 layers: the first is a dense feed-forward layer, the remaining 77 use MoE with 256 routed experts plus one shared expert, and eight routed experts fire per token. Attention is a gated variant of DeepSeek's sparse attention, with an index cache reused across layers to keep memory in check at long context. Two reasoning settings are exposed: "high" (the default, with a full chain of thought) and "no_think" for direct answers.

Simon Willison ran it through OpenRouter and noted the reasoning traces are written in slightly truncated English, which makes sense: nobody is grading the grammar of hidden reasoning tokens, so spending tokens on it would be waste.

Tencent's scoreboard, and who is holding the pen

Tencent's headline number comes from an internal blind evaluation: 163 experts scoring 203 engineering tasks, where Hy4 Preview averaged 2.99 out of 4.00 against 2.92 for GLM-5.3 and 2.94 for Kimi K3. The win rates are published too: 46.8% wins, 12.8% ties and 40.4% losses against GLM-5.3; 51.2% wins, 7.9% ties and 40.9% losses against Kimi K3. The model card also lists GPQA Diamond at 92.3, DeepSWE at 64.3 and SWE-Bench Pro at 65.7.

A blind panel is a more honest design than most vendor benchmark tables. It is still the vendor's panel. The margins are 0.05 to 0.07 of a point, and no independent lab has measured the model yet. When GLM-5.3-Flash and Qwen3.8-Flash-Next shipped last week we made the same point: launch-day numbers and third-party numbers have a habit of drifting apart.

Price and where to get it

Through Tencent Cloud TokenHub the API costs $0.834 per million input tokens and $2.501 per million output tokens, with cache hits at $0.042 per million. The model is wired into Tencent's WorkBuddy, CodeBuddy, Yuanbao and ima apps, free on WorkBuddy and CodeBuddy for two weeks, and is listed on OpenRouter. Self-hosting is a serious commitment: the deployment guides for vLLM and SGLang assume tensor parallelism across eight GPUs.

The decision this forces on buyers

Our view is that the important part of this release is not the 2.99. It is the combination of an Apache 2.0 license, a million-token window and a price under a dollar per million input tokens. That combination lets a company push an entire contract archive or a whole code repository through a single request, fine-tune without licensing negotiations, and do it cheaply. The catch is the 1.56 TB of weights and the eight-GPU floor, which put self-hosting out of reach for almost any small or mid-sized business; in practice, access means the API or a cloud provider that hosts the weights.

That routes the decision straight into data governance. Calling a Chinese-hosted API raises the question of where the data lands, and a marketplace layer such as OpenRouter does not fully answer it. Language coverage outside Chinese and English is also unmeasured; the model card documents those two only. If you run a multilingual product, the two-week free window is the cheap way to find out: build a small test set from your own tasks and trust that result over any table.

Sources: Tencent, Hugging Face model card, Simon Willison