# Qwen3.8-4B-Distill

> Qwen3.8-4B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture. The student was trained on ~45,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, general reasoning, and instruction following…

- **Organisation:** empero-ai
- **Category:** Chat & Text Generation, Reasoning Models
- **Parameters:** 4.7B
- **Context length:** 262,144 tokens
- **Licence:** Apache 2.0 (Commercial use OK)
- **Released:** 2026-08-15
- **Hugging Face:** 112.4K downloads (30 days), 74 likes

## Estimated memory

| Quant | ≈ GB |
|---|---|
| Q2_K | 3 |
| Q3_K_M | 3.73 |
| Q4_K_M | 4.26 |
| Q5_K_M | 4.74 |
| Q6_K | 5.25 |
| Q8_0 | 6.34 |
| F16 | 10.61 |

## Runs with

- [llama.cpp](https://selfhost.directory/project/llamacpp)
- [LM Studio](https://lmstudio.ai)
- [SGLang](https://selfhost.directory/project/sglang)
- [Transformers](https://github.com/huggingface/transformers)
- [vLLM](https://selfhost.directory/project/vllm)
- [Jan](https://jan.ai) (community)
- [KoboldCpp](https://selfhost.directory/project/koboldcpp) (community)
- [LocalAI](https://selfhost.directory/project/localai) (community)
- [MLX](https://github.com/ml-explore/mlx-lm) (community)
- [Ollama](https://selfhost.directory/project/ollama) (community)

## About

Qwen3.8-4B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture. The student was trained on ~45,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, general reasoning, and instruction following, quality-filtered before training.

The objective: bring the reasoning behavior of a frontier-scale teacher into a 4B that runs comfortably on consumer hardware.

• Distilled chain-of-thought — every answer opens with a block learned directly from Qwen3.8 2.4T A95B traces rather than synthetic self-generated reasoning.
• 4B weight class — bf16 fits in ~8 GB; quantized builds run on laptops and consumer GPUs.
• Native function calling per Qwen3.5's specification — no wrapper or tool-specific fine-tune required.
• 262,144-token native context, inherited from the Qwen3.5 base.
• Full fine-tune — every parameter updated; not an adapter.

• Type: Causal Language Model (text path of a vision-language base)
• Base: Qwen/Qwen3.5-4B
• Number of Parameters: 4B
• Training: SFT (off-policy distillation) on ~45,000 teacher traces
• Teacher: Qwen3.8 2.4T A95B (internal distillation datasets)
• Context Length: 262,144 natively

## Alternatives

- [nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16](https://selfhost.directory/models/nvidia/nvidia-nemotron-3-nano-4b-bf16) (4B, nvidia-nemotron-open-model-license, ≈4.17 GB at Q4): NVIDIA-Nemotron-3-Nano-4B-BF16 is a small language model (SLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. It responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response.
- [Qwen/Qwen3-4B-Instruct-2507](https://selfhost.directory/models/qwen/qwen3-4b-instruct-2507) (4B, Apache 2.0, ≈4.01 GB at Q4): We introduce the updated version of the Qwen3-4B non-thinking mode, named Qwen3-4B-Instruct-2507, featuring the following key enhancements:
- [Nanbeige/Nanbeige4.2-3B](https://selfhost.directory/models/nanbeige/nanbeige4.2-3b) (4.2B, Apache 2.0, ≈3.66 GB at Q4): Nanbeige4.2-3B is a compact agentic model built on Nanbeige4.2-3B-Base, designed to combine strong agentic behavior with broad reasoning and alignment capabilities. Its Looped Transformer architecture reuses the transformer layers to increase model capacity without adding parameters.
- [Qwen/Qwen3-4B](https://selfhost.directory/models/qwen/qwen3-4b) (4B, Apache 2.0, ≈4.01 GB at Q4): Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.
- [XHToken/Spark-X2.6-4B](https://selfhost.directory/models/xhtoken/spark-x2.6-4b) (4.1B, Apache 2.0, ≈4.06 GB at Q4): A little over a month has passed since Spark-X2.5. In the rapidly evolving world of AI agents, that is enough time to rethink, refine, and move the frontier forward.
- [ibm-granite/granite-4.1-3b](https://selfhost.directory/models/ibm-granite/granite-4.1-3b) (3.4B, Apache 2.0, ≈3.14 GB at Q4): Model Summary: Granite-4.1-3B is a 3B parameter long-context instruct model finetuned from Granite-4.1-3B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets.

More: https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/alternatives

## Runs well on

- [GeForce RTX 4060](https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/on/rtx-4060): Q8_0, est. 30 tok/s
- [Apple M4 (16 GB)](https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/on/apple-m4-16gb): F16, est. 7.1 tok/s
- [Radeon RX 6700 XT](https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/on/rx-6700-xt): F16, est. 23 tok/s
- [GeForce RTX 3060 12GB](https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/on/rtx-3060): F16, est. 21 tok/s
- [Arc B580](https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/on/arc-b580): F16, est. 27 tok/s
- [Tesla T4](https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill/on/tesla-t4): F16, est. 19 tok/s

## Similar size

- [Qwen/Qwen3-4B](https://selfhost.directory/models/qwen/qwen3-4b) (4B)
- [microsoft/Phi-4-mini-instruct](https://selfhost.directory/models/microsoft/phi-4-mini-instruct) (3.8B)
- [Qwen/Qwen3-4B-Instruct-2507](https://selfhost.directory/models/qwen/qwen3-4b-instruct-2507) (4B)
- [XHToken/Spark-X2.6-4B](https://selfhost.directory/models/xhtoken/spark-x2.6-4b) (4.1B)
- [ibm-granite/granite-4.2-3b](https://selfhost.directory/models/ibm-granite/granite-4.2-3b) (3.7B)
- [XHToken/Spark-X2.5-4B](https://selfhost.directory/models/xhtoken/spark-x2.5-4b) (4.1B)

## FAQ

**How much VRAM does Qwen3.8-4B-Distill need?**
About 4.3 GB at Q4_K_M with an 8K context. Q8_0 needs about 6.3 GB and full precision about 11 GB. Longer contexts need more memory for the KV cache.

**Can Qwen3.8-4B-Distill run on the RTX 4060?**
Yes. It fits at Q8_0 (about 6.0 GB), at an estimated 30 tokens/s. It fits comfortably on the Apple M4 (16 GB), Radeon RX 6700 XT, RTX 3060 12GB.

**Can I use Qwen3.8-4B-Distill commercially?**
Yes. It is released under Apache 2.0. Permissive: use, modify and sell freely; keep the notice and licence text.

**What is the context length of Qwen3.8-4B-Distill?**
262,144 tokens (256K). The full window needs far more memory than an 8K context.

**How do I run Qwen3.8-4B-Distill with Ollama or llama.cpp?**
With Ollama: ollama run hf.co/empero-ai/Qwen3.8-4B-Distill-GGUF:Q4_K_M. With llama.cpp: llama-server -hf empero-ai/Qwen3.8-4B-Distill-GGUF:Q4_K_M, which starts an OpenAI-compatible server.

## Links

- huggingface: https://huggingface.co/empero-ai/Qwen3.8-4B-Distill

---
Source: https://selfhost.directory/models/empero-ai/qwen3.8-4b-distill