# Hy4-base

> Hy4 is a new-generation Mixture-of-Experts (MoE) flagship model developed by the Tencent Hy Team, supporting text, image, and video understanding. The language backbone comprises 770B total parameters, of which 49B are activated per token.

- **Organisation:** tencent
- **Category:** Vision-Language Models
- **Parameters:** 771B total, 50.3B active (MoE)
- **Context length:** 1,048,576 tokens
- **Licence:** tencent-hy-license (Check licence)
- **Released:** 2026-11-18
- **Hugging Face:** 482 downloads (30 days), 9 likes

## Estimated memory

| Quant | ≈ GB |
|---|---|
| Q2_K | 249.63 |
| Q3_K_M | 370.28 |
| Q4_K_M | 458.89 |
| Q5_K_M | 538.07 |
| Q6_K | 622.9 |
| Q8_0 | 802.94 |
| F16 | 1509.9 |

## Runs with

- [Transformers](https://github.com/huggingface/transformers)
- [vLLM](https://selfhost.directory/project/vllm)

## About

Hy4 is a new-generation Mixture-of-Experts (MoE) flagship model developed by the Tencent Hy Team, supporting text, image, and video understanding. The language backbone comprises 770B total parameters, of which 49B are activated per token. It consists of 78 layers, with a standard dense FFN in the first layer and MoE FFNs in the remaining 77 layers. Each MoE layer contains 256 routed experts and 1 shared expert; every token activates the top-8 routed experts along with the shared expert. In addition to the backbone, a 5-layer DFly2 drafter (3.3B parameters) is built in for speculative decoding.

On the architecture side, inspired by DeepSeek and GLM, the attention module employs Gated DeepSeek Sparse Attention (Gated DSA) with IndexCache for cross-layer sparse index reuse. The residual pathway uses iHC (identity Hyper-Connections) to expand inter-layer information flow. A ViT supports dynamic image resolutions, with an adapter converting image and video frame features into visual tokens for the language model to process alongside text tokens. DFly2, a parallel drafter, reuses Hy4's KV cache and sparse indices to support long-context speculative decoding at a low, constant cost.

Bits-per-byte (BPB) on internal code & agent validation, lower is better:

Few-shot pretrain benchmark evaluation:

## Alternatives

- [deepseek-ai/DeepSeek-V4.1-Flash](https://selfhost.directory/models/deepseek-ai/deepseek-v4.1-flash) (763B (188B active), MIT, ≈453.59 GB at Q4): We introduce DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens. The model natively processes images and text, and generates text autoregressively.
- [deepseek-ai/DeepSeek-V4.1-Flash-1017](https://selfhost.directory/models/deepseek-ai/deepseek-v4.1-flash-1017) (763B (188B active), MIT, ≈453.59 GB at Q4): DeepSeek-V4.1-Flash-1017 is a minor update to DeepSeek-V4.1-Flash, delivering substantial improvements across reasoning, coding, agentic tasks, and visual benchmarks.
- [thinkingmachines/Inkling](https://selfhost.directory/models/thinkingmachines/inkling) (952B (18.1B active), Apache 2.0, ≈567.17 GB at Q4): Inkling is a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages.
- [moonshotai/Kimi-K2.7-Code](https://selfhost.directory/models/moonshotai/kimi-k2.7-code) (1.03T (33.3B active), modified-mit, ≈632.16 GB at Q4): Kimi K2.7 Code is a coding-focused agentic model built upon Kimi K2.6. With substantial improvements on real-world long-horizon coding tasks, it strengthens end-to-end task completion across complex software engineering workflows while improving token efficiency, reducing thinking-token usage by…
- [meta-models/Muse-Spark-Open-816B-A42B](https://selfhost.directory/models/meta-models/muse-spark-open-816b-a42b) (822B (42B active), muse-model-license, ≈489.78 GB at Q4): Authors: Meta Superintelligence Labs Model Release Date: October 2026 License: Muse Model License
- [moonshotai/Kimi-K2.6](https://selfhost.directory/models/moonshotai/kimi-k2.6) (1.03T (33.3B active), modified-mit, ≈632.16 GB at Q4): Kimi K2.6 is an open-source, native multimodal agentic model that advances practical capabilities in long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration.

More: https://selfhost.directory/models/tencent/hy4-base/alternatives

## Similar size

- [deepseek-ai/DeepSeek-V4.1-Flash-1017](https://selfhost.directory/models/deepseek-ai/deepseek-v4.1-flash-1017) (763B)
- [deepseek-ai/DeepSeek-V4.1-Flash](https://selfhost.directory/models/deepseek-ai/deepseek-v4.1-flash) (763B)
- [mistralai/Mistral-Large-4-1T-A52B](https://selfhost.directory/models/mistralai/mistral-large-4-1t-a52b) (1T)
- [thinkingmachines/Inkling](https://selfhost.directory/models/thinkingmachines/inkling) (952B)
- [meta-models/Muse-Spark-Open-816B-A42B](https://selfhost.directory/models/meta-models/muse-spark-open-816b-a42b) (822B)
- [tencent/Hy4](https://selfhost.directory/models/tencent/hy4) (774B)

## FAQ

**How much VRAM does Hy4-base need?**
About 459 GB at Q4_K_M with an 8K context. Q8_0 needs about 803 GB and full precision about 1510 GB. Longer contexts need more memory for the KV cache.

**Can Hy4-base run on a consumer GPU?**
Not comfortably on a single one: it needs about 459 GB at Q4. Split it over several GPUs, use a large unified-memory machine, or offload part of it to system RAM.

**Can I use Hy4-base commercially?**
Unclear. It is released under tencent-hy-license. Custom licence: read the model card before using it commercially. Check the licence on the model card before using it in a product.

**What is the context length of Hy4-base?**
1,048,576 tokens (1M). The full window needs far more memory than an 8K context.

**How do I run Hy4-base locally?**
With vLLM: vllm serve tencent/Hy4-base.

## Links

- huggingface: https://huggingface.co/tencent/Hy4-base
- homepage: https://hy.tencent.ai/research/hy4
- github: https://github.com/Tencent-Hunyuan/Hy4
- paper: https://arxiv.org/abs/2603.12201

---
Source: https://selfhost.directory/models/tencent/hy4-base