selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

internlm logo

InternLumina-U2

internlm·Instruct·released 1 Sept 2026

Apache 2.016B≈11 GB
Your GPU
hf download internlm/InternLumina-U2
Setup ↓

A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding, Image Generation and Editing.

Notify me when

How much VRAM does InternLumina-U2 need?

At Q4_K_M InternLumina-U2 needs about 11 GB with an 8K context (11 GB at Q4, 33 GB at full precision). Pick your GPU to see what fits and how fast it runs.

--vram
any
--gpus
--quant

Needs: ≈11 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.

Change quant, context or KV cache to compare
Advanced: context 8K · KV cache 16-bit
Context
KV cache

Smallest GPU for each quant and context (KV cache 16-bit)

Quantisation4K8K16K32K64K128K
Q3_K_M
Q4_K_M
Q5_K_M
Q6_K
Q8_0
FP16

≤16 GB card24–32 GB48–96 GBmore

QuantisationQualityWeightsKV cacheTotal
Q2_KHeavy lossSmallest, noticeably worse qualityHeavy loss4.9 GB 1.4 GB*≈7.1 GB
Q3_K_MNoticeable lossSmall, some quality lossNoticeable loss7.3 GB 1.4 GB*≈9.6 GB
Q4_K_MrecommendedSmall lossRecommended default for local useSmall loss9.0 GB 1.4 GB*≈11 GB
Q5_K_MNear-losslessNear-lossless for most tasksNear-lossless11 GB 1.4 GB*≈13 GB
Q6_KNear-losslessPractically losslessNear-lossless12 GB 1.4 GB*≈15 GB
Q8_0LosslessLossless in practice, twice the size of Q4Lossless16 GB 1.4 GB*≈19 GB
FP16 / BF16LosslessFull precision, as releasedLossless30 GB 1.4 GB*≈33 GB

Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it.

How to pick a quant

Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.

  • LosslessF16, Q8_0
  • Near-losslessQ6_K, Q5_K_M
  • Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
  • Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
  • Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K

Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B

How much context fits?

Model quant
KV cache

At 8K context, Q4_K_M needs ≈11 GB(9.0 GB weights + 1.4 GB KV cache + 0.9 GB overhead). Pick a GPU above to see how much context fits.

0102030401K2K4K8K16K32K64K128KGB

Weights + overheadKV cache

Can the Tesla T4 run InternLumina-U2? →

Speed vs quality per quant

Smaller quants run faster and fit in less memory, but lose quality.

Q2_KQ3_K_MQ4_K_MQ5_K_MQ6_KQ8_0F16

Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.

Real-world speed of InternLumina-U2

Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →

No medians yet. Be one of the first 3 to report a setup.

All reports

Run InternLumina-U2 locally

About InternLumina-U2

A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding, Image Generation and Editing. 📑 Paper · 🌐 Project Page · 💻 Code Intern Lumina U2 is a unified multimodal model that brings language, image, video and 3D into a single framework, covering text QA, text-to-image generation, image understanding, image editing, video understanding and 3D understanding with one model. It is a 16B-parameter MoE with 1B active parameters (16B-A1B), pairing an efficient sparse backbone with an 8-codebook fully-discrete visual representation built on AToken. This repo hosts checkpoints of the same architecture trained on different hardware stacks: • ascend/ — trained on Huawei Ascend NPUs. • nvidia/ — trained on NVIDIA GPUs.

Full model card on Hugging Face →

Alternatives to InternLumina-U2

All alternatives →
google logo

gemma-4-12B-it

google·Omni

≈11 GB

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants.

12B256K ctx Apache 2.0
1.6MMay 2026ollama
google logo

gemma-4-E4B-it

google·Omni

≈5.9 GB

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants.

8B128K ctx Apache 2.0
4.3MMar 2026ollama
sensenova logo

SenseNova-U1.5-8B-MoT

sensenova·Omni

≈12 GB

SenseNova-U1.5-8B-MoT is our latest native unified multimodal checkpoint for more accurate, consistent, reliable, and aesthetically compelling visual creation.

17.5B256K ctx Apache 2.0
6.5KAug 2026gguf
google logo

gemma-4-12B

google·Omni

≈11 GB

Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants.

12B256K ctx Apache 2.0
48.8KMay 2026gguf
inclusionAI logo

LLaDA2.0-Uni

inclusionAI·Omni

≈10 GB

LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Model

16.3BA1.5B8K ctx Apache 2.0
2.2KApr 2026
nvidia logo

NLD-Uni

nvidia·Omni

≈12 GB

NLD-Uni is a unified multimodal model for image understanding and image generation. It combines a diffusion language model with specialized visual components and a mixture-of-transformers design.

18.3B128K ctx Unknown licence
1.3KOct 2026

InternLumina-U2 FAQ

How much VRAM does InternLumina-U2 need?
About 11 GB at Q4_K_M with an 8K context. Q8_0 needs about 19 GB and full precision about 33 GB. Longer contexts need more memory for the KV cache.
Can InternLumina-U2 run on the Tesla T4?
Yes. It fits at Q6_K (about 15 GB), at an estimated 13 tokens/s. It fits comfortably on the Arc A770 16GB, Radeon RX 7800 XT, Tesla P40.
Can I use InternLumina-U2 commercially?
Yes. It is released under Apache 2.0. Permissive: use, modify and sell freely; keep the notice and licence text.

Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.