Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance.
ZwZ2-9B
inclusionAI·Instruct·released 16 Oct 2026
llama-server -hf inclusionAI/ZwZ2-9B-GGUF:Q4_K_MZwZ2-9B is the Qwen3.5-9B-based model in Zooming-without-Zooming-v2 (ZwZ2). It learns to connect visual evidence across regions through comparison, aggregation, matching, and multi-step reasoning. Training uses full images; inference requires no zooming tools.
How much VRAM does ZwZ2-9B need?
At Q4_K_M ZwZ2-9B needs about 7.1 GB with an 8K context (7.1 GB at Q4, 20 GB at full precision). Pick your GPU to see what fits and how fast it runs.
Needs: ≈7.0 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.
Advanced: context 8K · KV cache 16-bit
Smallest GPU for each quant and context (KV cache 16-bit)
| Quantisation | 8K | 16K | 32K | 64K | 128K | 256K |
|---|---|---|---|---|---|---|
| Q3_K_M | ||||||
| Q4_K_M | ||||||
| Q5_K_M | ||||||
| Q6_K | ||||||
| Q8_0 | ||||||
| FP16 |
≤16 GB card24–32 GB48–96 GBmore
| Quantisation | Total |
|---|---|
| Q2_KHeavy lossSmallest, noticeably worse quality | ≈4.5 GB |
| Q3_K_MNoticeable lossSmall, some quality loss | ≈6.0 GB |
| Q4_K_MrecommendedSmall lossRecommended default for local use | ≈7.0 GB |
| Q5_K_MNear-losslessNear-lossless for most tasks | ≈7.8 GB |
| Q6_KNear-losslessPractically lossless | ≈8.7 GB |
| Q8_0LosslessLossless in practice, twice the size of Q4 | ≈11 GB |
| FP16 / BF16LosslessFull precision, as released | ≈19 GB |
Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it. Weights marked “file” use the real GGUF download size; “est.” rows are computed from the parameter count.
How to pick a quant
Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.
- LosslessF16, Q8_0
- Near-losslessQ6_K, Q5_K_M
- Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
- Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
- Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K
Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B
How much context fits?
At 8K context, Q4_K_M needs ≈7.0 GB(5.2 GB weights + 1.0 GB KV cache + 0.8 GB overhead). Pick a GPU above to see how much context fits.
Weights + overheadKV cache
Can the RTX 4060 run ZwZ2-9B? →
Speed vs quality per quant
Smaller quants run faster and fit in less memory, but lose quality.
Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.
Real-world speed of ZwZ2-9B
Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →
No medians yet. Be one of the first 3 to report a setup.
All reports
Run ZwZ2-9B locally
--quantQ4_K_Mdefault; pick your GPU to tailor itinclusionAI/ZwZ2-9B-GGUF
llama-server -hf inclusionAI/ZwZ2-9B-GGUF:Q4_K_MDownloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.
vllm serve inclusionAI/ZwZ2-9Bservices:
vllm:
image: vllm/vllm-openai:latest
command: inclusionAI/ZwZ2-9B
ports:
- "8000:8000"
ipc: host
volumes:
- ~/.cache/huggingface:/root/.cache/huggingface
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]API on port 8000.
sglang serve inclusionAI/ZwZ2-9B --host 0.0.0.0 --port 30000services:
sglang:
image: lmsysorg/sglang:latest
command: sglang serve inclusionAI/ZwZ2-9B --host 0.0.0.0 --port 30000
ports:
- "30000:30000"
ipc: host
volumes:
- ~/.cache/huggingface:/root/.cache/huggingface
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]API on port 30000.
from transformers import pipeline
pipe = pipeline("image-text-to-text", model="inclusionAI/ZwZ2-9B", dtype="auto", device_map="auto")
messages = [
{
"role": "user",
"content": [
{"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"},
{"type": "text", "text": "Describe this image."},
],
}
]
out = pipe(text=messages, max_new_tokens=256)
print(out[0]["generated_text"][-1]["content"])Quantised versions and downloads
| Repository | Format | Quant | Size |
|---|---|---|---|
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q3_K_M | 4.6 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q3_K_L | 4.9 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q4_K_S | 5.4 GB |
| inclusionAI/ZwZ2-9B-GGUF | GGUF | Q4_K_M | 5.6 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q4_K_M | 5.6 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q5_K_S | 6.3 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q5_K_M | 6.5 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q6_K | 7.4 GB |
| inclusionAI/ZwZ2-9B-GGUF | GGUF | Q8_0 | 9.5 GB |
| prithivMLmods/inclusionAI-ZwZ2-9B-GGUF | GGUF | Q8_0 | 9.5 GB |
About ZwZ2-9B
ZwZ2-9B is the Qwen3.5-9B-based model in Zooming-without-Zooming-v2 (ZwZ2). It learns to connect visual evidence across regions through comparison, aggregation, matching, and multi-step reasoning. Training uses full images; inference requires no zooming tools. On 25 benchmarks across six domains, it improves over its base model on every benchmark, increasing overall accuracy from 51.63 to 56.59 (+4.96 points). Gains extend from fine-grained perception to spatial reasoning, long-context understanding, multimodal agents, and general visual reasoning. EvidenceConnect builds verified multi-region questions from SA-1B images: • Group 3–7 related regions and generate questions using privileged zoomed views. • Check answers through cross-model agreement and drop-one region tests. • Convert verified questions to four-option tasks and train with GRPO, without an additional supervised fine-tuning stage. All three model sizes use approximately 20K training examples. The policy sees full images with colored region outlines; privileged crop layouts are used only for data construction. See ZwZ2-RL-VQA for the data schema and synthesis process. Editable vector PDF. The figure shows eight representative benchmarks; the complete results follow.
Full model card on Hugging Face →Other ZwZ2 models
Whole family →Alternatives to ZwZ2-9B
All alternatives →Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.
ZAYA1-VL-8B is a vision-language model (VLM) built upon Zyphra's ZAYA1-8B LLM. It has state-of-the-art performance among VLMs for its size and inference efficiency.
Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance.
MiMo-V2.6-Distill-Qwen-9B is a 9B agentic model developed by Xiaomi MiMo through supervised fine-tuning of Qwen3.5-9B on MiMo-generated data. It covers coding, general-purpose agent tasks, visual coding, and cybersecurity.
FLUX 3 Image Klein Prompt Upsampler is a 9B vision-language model that expands a short image request into the detailed prompt FLUX 3 Image Klein works best with. It takes the request, up to 10 reference images and an optional list of bounding boxes, and returns one prompt.
ZwZ2-9B FAQ
- How much VRAM does ZwZ2-9B need?
- About 7.1 GB at Q4_K_M with an 8K context. Q8_0 needs about 11 GB and full precision about 20 GB. Longer contexts need more memory for the KV cache.
- Can ZwZ2-9B run on the RTX 4060?
- Yes. It fits at Q4_K_M (about 7.0 GB), at an estimated 26 tokens/s. It fits comfortably on the Apple M4 (16 GB), Radeon RX 6700 XT, RTX 3060 12GB.
- Can I use ZwZ2-9B commercially?
- Unclear. It is released under Unknown licence. No licence stated: assume all rights reserved. Check the licence on the model card before using it in a product.
- What is the context length of ZwZ2-9B?
- 262,144 tokens (256K). The full window needs far more memory than an 8K context.
- How do I run ZwZ2-9B with llama.cpp?
- Run llama-server -hf inclusionAI/ZwZ2-9B-GGUF:Q4_K_M. It downloads the GGUF and starts an OpenAI-compatible server.
Rate this model
Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.
0 comments