selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

Best LLMs and AI models for 12 GB VRAM (2026)

2 340 open-weight models fit fully in 12 GB of GPU memory at their recommended quantisation. Each section puts the strongest model that still fits comfortably first, with the best quantisation and an estimated generation speed.

A GPU with 12 GB of VRAM has 12 GB usable for models, which covers open-weight models from about 0.8B to 12B parameters. Several popular models fit that budget, including Qwen3-8B from Qwen, Llama-3.1-8B-Instruct from meta-llama and gemma-4-12B-it from google. Smaller options such as Qwen3-0.6B also fit, along with gemma-4-E4B-it and Qwen3.5-9B.

Best chat models for 12 GB

All 874 that fit →
#ModelParamsBest quantMemorySpeedLicence
18.2BQ8_0≈10 GBFits in memory~27 tok/sCommercial OK
214.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
3
ornith-ai logoOrnith-1.5-9Bornith-ai
9.7BQ8_0≈11 GBFits in memory~23 tok/sCommercial OK
4
inclusionAI logoLing-3.0-tinyinclusionAI
7.9BA1.4BQ8_0≈10 GBFits in memory~162 tok/sCommercial OK
514.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
69BQ8_0≈11 GBFits in memory~25 tok/sCommercial OK
7
microsoft logophi-4microsoft
14.7BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
8
ibm-granite logogranite-4.2-8bibm-granite
8.8BQ8_0≈11 GBFits in memory~25 tok/sCommercial OK

Best code models for 12 GB

All 74 that fit →
#ModelParamsBest quantMemorySpeedLicence
112.1BA2.5BQ5_K_M≈11 GBFits in memory~134 tok/sCommercial OK
212.1BA2.5BQ6_K≈11 GBFits in memory~115 tok/sCommercial OK
314.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
4
Tesslate logoOmniCoder-9BTesslate
9.4BQ8_0≈11 GBFits in memory~24 tok/sCommercial OK
57.6BQ8_0≈8.9 GBFits in memory~29 tok/sCommercial OK
616BQ4_K_M≈11 GBFits in memory~25 tok/sConditions
7
NousResearch logoNousCoder-14BNousResearch
14.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
814.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK

Best reasoning models for 12 GB

All 50 that fit →
#ModelParamsBest quantMemorySpeedLicence
114.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
214.7BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
314.7BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
47.3BQ6_K≈10 GBFits in memory~40 tok/sCommercial OK
58.6BQ8_0≈10 GBFits in memory~26 tok/sCommercial OK
614.8BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
7
inclusionAI logoRing-mini-2.0inclusionAI
16.3BA1.4BQ4_K_M≈11 GBFits in memory~273 tok/sCommercial OK
87.6BQ8_0≈8.9 GBFits in memory~29 tok/sCommercial OK

Best vision models for 12 GB

All 331 that fit →
#ModelParamsBest quantMemorySpeedLicence
19.7BQ8_0≈11 GBFits in memory~23 tok/sCommercial OK
2
Cloudflare logoclef-flashCloudflare
9.4BQ8_0≈11 GBFits in memory~24 tok/sCommercial OK
38.8BQ8_0≈10 GBFits in memory~26 tok/sCommercial OK
49.7BQ8_0≈11 GBFits in memory~23 tok/sCommercial OK
5
black-forest-labs logoflux-3-image-klein-pupblack-forest-labs
9.4BQ8_0≈11 GBFits in memory~24 tok/sCommercial OK
69.4BQ8_0≈11 GBFits in memory~24 tok/sCommercial OK
74.7BQ8_0≈5.9 GBFits in memory~48 tok/sCommercial OK
89.8BQ8_0≈11 GBFits in memory~23 tok/sCheck licence

Best image gen models for 12 GB

All 99 that fit →
#ModelParamsBest quantMemoryLicence
17.1BQ8_0≈8.9 GBFits in memoryNon-commercial
27.1BQ8_0≈8.9 GBFits in memoryNon-commercial
3
Tongyi-MAI logoZ-Image-TurboTongyi-MAI
6.2BQ8_0≈7.8 GBFits in memoryCommercial OK
4
OpenMOSS-Team logoMOSS-Image-SFTOpenMOSS-Team
5.1BQ8_0≈6.5 GBFits in memoryCommercial OK
5
OpenMOSS-Team logoMOSS-Image-BaseOpenMOSS-Team
5.1BQ8_0≈6.5 GBFits in memoryCommercial OK
6
Tongyi-MAI logoZ-ImageTongyi-MAI
6.2BQ8_0≈7.8 GBFits in memoryCommercial OK
78.1BQ8_0≈10 GBFits in memoryNon-commercial
86.2BQ8_0≈7.8 GBFits in memoryCommercial OK

Best STT models for 12 GB

All 99 that fit →
#ModelParamsBest quantMemoryLicence
12.1BFP16 / BF16≈5.1 GBFits in memoryCommercial OK
22.3BFP16 / BF16≈5.8 GBFits in memoryCommercial OK
38.7BQ8_0≈11 GBFits in memoryCommercial OK
4
microsoft logoVibeVoice-ASRmicrosoft
8.7BQ8_0≈11 GBFits in memoryCommercial OK
54.4BFP16 / BF16≈10 GBFits in memoryCommercial OK
62.3BFP16 / BF16≈5.7 GBFits in memoryCommercial OK
7909MFP16 / BF16≈2.5 GBFits in memoryCommercial OK
8627MFP16 / BF16≈1.9 GBFits in memoryCommercial OK

Best TTS models for 12 GB

All 109 that fit →
#ModelParamsBest quantMemoryLicence
1
m-a-p logoYuE2-3Bm-a-p
3.6BFP16 / BF16≈8.6 GBFits in memoryNon-commercial
2
BreezeBlue logoBreeze-TTS-2BreezeBlue
3.5BFP16 / BF16≈8.3 GBFits in memoryNon-commercial
3
openbmb logoVoxCPM2openbmb
2.3BFP16 / BF16≈5.6 GBFits in memoryCommercial OK
41.9BFP16 / BF16≈4.8 GBFits in memoryCommercial OK
5
k2-fsa logoOmniVoicek2-fsa
613MFP16 / BF16≈1.9 GBFits in memoryCheck licence
61.5BFP16 / BF16≈3.8 GBFits in memoryCommercial OK
7915MFP16 / BF16≈2.5 GBFits in memoryCommercial OK
8
fishaudio logos2-profishaudio
4.6BFP16 / BF16≈11 GBFits in memoryNon-commercial

Best embedding models for 12 GB

All 238 that fit →
#ModelParamsBest quantMemoryLicence
1744MFP16 / BF16≈2.2 GBFits in memoryCommercial OK
27.6BQ8_0≈9.5 GBFits in memoryCommercial OK
3303MFP16 / BF16≈1.2 GBFits in memoryConditions
4596MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
54BFP16 / BF16≈9.5 GBFits in memoryCommercial OK
68BQ8_0≈9.9 GBFits in memoryCommercial OK
78.1BQ8_0≈10 GBFits in memoryCommercial OK
81.1BFP16 / BF16≈3.0 GBFits in memoryCommercial OK

GPUs with 12 GB 32

Frequently asked questions

What is the best LLM for 12 GB of VRAM?
Our top chat pick for 12 GB is Qwen3-8B (Qwen) at Q8_0, needing about 10.15 GB, at roughly 27 tokens/s. The ranking favours the strongest model that still fits comfortably, then benchmarks, popularity and release date.
What is the best coding model for 12 GB of VRAM?
Mellum2.1-12B-A2.5B-Thinking (JetBrains) at Q5_K_M, needing about 10.68 GB, at roughly 134 tokens/s.
How big a model fits in 12 GB of VRAM?
Dense LLMs up to about 16B parameters fit fully in 12 GB at Q4_K_M or better, with an 8K-token context. Longer contexts need more memory for the KV cache.
How many open-weight models run on 12 GB of VRAM?
2340 open-weight models on this site fit fully in 12 GB at their recommended quantisation, across 19 categories (chat, code, vision, image generation, speech and more).
How fast are models on 12 GB of VRAM?
Token generation is limited by memory bandwidth. The speeds on this page assume about 432 GB/s, typical for 12 GB cards; a faster or slower card scales roughly in proportion. Open your GPU’s page for estimates with its own bandwidth.
Which GPUs have 12 GB of VRAM?
Among others: GeForce RTX 5070, Arc B580, GeForce RTX 4070, GeForce RTX 3060 12GB, Radeon RX 6700 XT, GeForce RTX 5070 Ti Laptop. Each GPU page lists what fits and how fast it runs.
What if a model doesn’t fit in 12 GB?
llama.cpp, Ollama and LM Studio can offload part of an LLM to system RAM. It runs, but generation slows down a lot because RAM is far slower than VRAM. The 16 GB page shows what one more step of memory buys.

Memory figures are estimates: weights at the given quantisation plus an 8K-token KV cache and runtime overhead. On Macs and other unified-memory machines only part of the memory is available to the GPU, see the Apple Silicon pages.