selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

AI models that run on the GeForce RTX 3060 12GB

With 12 GB of VRAM, the GeForce RTX 3060 12GB can run 2 343 open-weight models fully in memory at their recommended quantisation. Each section puts the strongest models that fit comfortably first, with the best quantisation that still fits, and an estimated generation speed.

The GeForce RTX 3060 12GB has 12 GB of GDDR6 memory, and all 12 GB is usable for models. Its 360 GB/s memory bandwidth affects how quickly a model can produce output, so memory size determines which models can load and bandwidth determines how fast they respond. The card supports CUDA and Vulkan backends. Popular open-weight models that fit include gemma-4-E4B-it, Llama-3.1-8B-Instruct, and Qwen3-8B.

VRAM
12 GB
usable for models
≈12 GB
bandwidth
360 GB/s
backends
cuda, vulkan
memory
GDDR6 · 192-bit
power
170 W
vendor · year
NVIDIA · 2021
price · value ranking →
$329launch MSRP, 2021

Specs: Wikipedia (CC BY-SA 4.0)

ModelParamsBest quantMemorySpeedLicence
8.2BQ8_0≈10 GBFits in memory~23 tok/sCommercial OK
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
ornith-ai logoOrnith-1.5-9Bornith-ai
9.7BQ8_0≈11 GBFits in memory~19 tok/sCommercial OK
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
inclusionAI logoLing-3.0-tinyinclusionAI
7.9BA1.4BQ8_0≈10 GBFits in memory~135 tok/sCommercial OK
9.4BQ8_0≈11 GBFits in memory~20 tok/sCommercial OK
9BQ8_0≈11 GBFits in memory~21 tok/sCommercial OK
microsoft logophi-4microsoft
14.7BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
ibm-granite logogranite-4.2-8bibm-granite
8.8BQ8_0≈11 GBFits in memory~21 tok/sCommercial OK
9.2BQ6_K≈11 GBFits in memory~26 tok/sConditions
ibm-granite logogranite-4.2-3bibm-granite
3.7BQ8_0≈4.9 GBFits in memory~51 tok/sCommercial OK
7.6BQ8_0≈8.9 GBFits in memory~24 tok/sCommercial OK
ModelParamsBest quantMemorySpeedLicence
12.1BA2.5BQ5_K_M≈11 GBFits in memory~111 tok/sCommercial OK
12.1BA2.5BQ6_K≈11 GBFits in memory~96 tok/sCommercial OK
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
Tesslate logoOmniCoder-9BTesslate
9.4BQ8_0≈11 GBFits in memory~20 tok/sCommercial OK
7.6BQ8_0≈8.9 GBFits in memory~24 tok/sCommercial OK
16BQ4_K_M≈11 GBFits in memory~20 tok/sConditions
NousResearch logoNousCoder-14BNousResearch
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
4BQ8_0≈7.5 GBFits in memory~46 tok/sCommercial OK
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
8.3BQ8_0≈10 GBFits in memory~23 tok/sCommercial OK
8.5BQ6_K≈11 GBFits in memory~28 tok/sConditions
ModelParamsBest quantMemorySpeedLicence
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
14.7BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
14.7BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
7.3BQ6_K≈10 GBFits in memory~33 tok/sCommercial OK
8.6BQ8_0≈10 GBFits in memory~22 tok/sCommercial OK
14.8BQ4_K_M≈11 GBFits in memory~22 tok/sCommercial OK
inclusionAI logoRing-mini-2.0inclusionAI
16.3BA1.4BQ4_K_M≈11 GBFits in memory~228 tok/sCommercial OK
14.7BQ4_K_M≈10 GBFits in memory~22 tok/sConditions
7.6BQ8_0≈8.9 GBFits in memory~24 tok/sCommercial OK
8.2BQ8_0≈10 GBFits in memory~23 tok/sCommercial OK
7.3BQ6_K≈10 GBFits in memory~33 tok/sCommercial OK
8BQ8_0≈9.8 GBFits in memory~23 tok/sCommercial OK
ModelParamsBest quantMemorySpeedLicence
9.7BQ8_0≈11 GBFits in memory~19 tok/sCommercial OK
Cloudflare logoclef-flashCloudflare
9.4BQ8_0≈11 GBFits in memory~20 tok/sCommercial OK
8.8BQ8_0≈10 GBFits in memory~21 tok/sCommercial OK
9.7BQ8_0≈11 GBFits in memory~19 tok/sCommercial OK
black-forest-labs logoflux-3-image-klein-pupblack-forest-labs
9.4BQ8_0≈11 GBFits in memory~20 tok/sCommercial OK
4.7BQ8_0≈5.9 GBFits in memory~40 tok/sCommercial OK
9.4BQ8_0≈11 GBFits in memory~20 tok/sCommercial OK
12.2BQ4_K_M≈10 GBFits in memory~27 tok/sConditions
9.8BQ8_0≈11 GBFits in memory~19 tok/sCheck licence
9.4BQ8_0≈11 GBFits in memory~20 tok/sNon-commercial
microsoft logoAesCode-8Bmicrosoft
8.8BQ8_0≈10 GBFits in memory~21 tok/sCommercial OK
inclusionAI logoUI-Venus-2-9BinclusionAI
9BQ8_0≈11 GBFits in memory~21 tok/sCheck licence
ModelParamsBest quantMemorySpeedLicence
12BQ4_K_M≈10 GBFits in memory~27 tok/sCommercial OK
8BQ8_0≈9.2 GBFits in memory~23 tok/sCommercial OK
9.4BQ8_0≈10 GBFits in memory~20 tok/sCommercial OK
12BQ4_K_M≈11 GBFits in memory~27 tok/sCommercial OK
tencent logoWeVisCuttencent
9.4BQ8_0≈11 GBFits in memory~20 tok/sCheck licence
9.1BQ8_0≈10 GBFits in memory~21 tok/sCommercial OK
5.1BQ8_0≈5.7 GBFits in memory~36 tok/sCommercial OK
8.7BQ8_0≈8.9 GBFits in memory~21 tok/sCommercial OK
inclusionAI logoLLaDA2.0-UniinclusionAI
16.3BA1.5BQ4_K_M≈10 GBFits in memory~218 tok/sCommercial OK
7.8BQ8_0≈8.2 GBFits in memory~24 tok/sConditions
ByteDance-Seed logoBAGEL-7B-MoTByteDance-Seed
14.7BQ5_K_M≈11 GBFits in memory~19 tok/sCommercial OK
LiquidAI logoLFM3-Omni-2BLiquidAI
2.2BQ8_0≈2.7 GBFits in memory~86 tok/sConditions
ModelParamsBest quantMemoryLicence
3.3BA1.1BFP16 / BF16≈8.0 GBFits in memoryCommercial OK
4.5BFP16 / BF16≈11 GBFits in memoryCommercial OK
datalab-to logochandra-ocr-2datalab-to
5.3BQ8_0≈6.8 GBFits in memoryConditions
zai-org logoGLM-OCRzai-org
1.3BFP16 / BF16≈3.5 GBFits in memoryCommercial OK
3.3BA1.1BFP16 / BF16≈8.0 GBFits in memoryCommercial OK
PaddlePaddle logoPaddleOCR-VL-1.6PaddlePaddle
959MFP16 / BF16≈2.6 GBFits in memoryCommercial OK
853MFP16 / BF16≈2.4 GBFits in memoryCommercial OK
XingChen-AGI logoTeleOCRXingChen-AGI
1.4BFP16 / BF16≈3.7 GBFits in memoryCommercial OK
1.3BFP16 / BF16≈3.4 GBFits in memoryCommercial OK
tencent logoHunyuanOCRtencent
1.1BFP16 / BF16≈3.0 GBFits in memoryConditions
2.2BFP16 / BF16≈5.5 GBFits in memoryCommercial OK
1.2BFP16 / BF16≈3.1 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
2.1BFP16 / BF16≈5.1 GBFits in memoryCommercial OK
2.3BFP16 / BF16≈5.8 GBFits in memoryCommercial OK
8.7BQ8_0≈11 GBFits in memoryCommercial OK
microsoft logoVibeVoice-ASRmicrosoft
8.7BQ8_0≈11 GBFits in memoryCommercial OK
4.4BFP16 / BF16≈10 GBFits in memoryCommercial OK
2.3BFP16 / BF16≈5.7 GBFits in memoryCommercial OK
627MFP16 / BF16≈1.9 GBFits in memoryCommercial OK
909MFP16 / BF16≈2.5 GBFits in memoryCommercial OK
netease-youdao logoConfucius4-R2T2netease-youdao
2BFP16 / BF16≈5.1 GBFits in memoryCheck licence
938MFP16 / BF16≈2.6 GBFits in memoryCommercial OK
2.1BFP16 / BF16≈5.1 GBFits in memoryCommercial OK
600MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
m-a-p logoYuE2-3Bm-a-p
3.6BFP16 / BF16≈8.6 GBFits in memoryNon-commercial
BreezeBlue logoBreeze-TTS-2BreezeBlue
3.5BFP16 / BF16≈8.3 GBFits in memoryNon-commercial
openbmb logoVoxCPM2openbmb
2.3BFP16 / BF16≈5.6 GBFits in memoryCommercial OK
1.9BFP16 / BF16≈4.8 GBFits in memoryCommercial OK
k2-fsa logoOmniVoicek2-fsa
613MFP16 / BF16≈1.9 GBFits in memoryCheck licence
1.5BFP16 / BF16≈3.8 GBFits in memoryCommercial OK
915MFP16 / BF16≈2.5 GBFits in memoryCommercial OK
fishaudio logos2-profishaudio
4.6BFP16 / BF16≈11 GBFits in memoryNon-commercial
8.9BA2.1BQ8_0≈11 GBFits in memoryCommercial OK
hexgrad logoKokoro-82Mhexgrad
82MFP16 / BF16≈0.7 GBFits in memoryCommercial OK
601MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
OpenMOSS-Team logoMOSS-TTS-v1.5OpenMOSS-Team
8.5BQ8_0≈11 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
2.4BFP16 / BF16≈5.9 GBFits in memoryCheck licence
8.4BQ8_0≈10 GBFits in memoryCommercial OK
2.3BFP16 / BF16≈5.7 GBFits in memoryConditions
5BQ8_0≈6.4 GBFits in memoryCommercial OK
1.5BFP16 / BF16≈3.8 GBFits in memoryConditions
1.5BFP16 / BF16≈3.8 GBFits in memoryConditions
568MFP16 / BF16≈1.8 GBFits in memoryConditions
5BQ8_0≈6.4 GBFits in memoryCommercial OK
568MFP16 / BF16≈1.8 GBFits in memoryConditions
1.4BFP16 / BF16≈3.7 GBFits in memoryCommercial OK
171MFP16 / BF16≈0.9 GBFits in memoryCommercial OK
2.3BFP16 / BF16≈5.7 GBFits in memoryConditions
ModelParamsBest quantMemoryLicence
744MFP16 / BF16≈2.2 GBFits in memoryCommercial OK
7.6BQ8_0≈9.5 GBFits in memoryCommercial OK
303MFP16 / BF16≈1.2 GBFits in memoryConditions
596MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
4BFP16 / BF16≈9.5 GBFits in memoryCommercial OK
8BQ8_0≈9.9 GBFits in memoryCommercial OK
8.1BQ8_0≈10 GBFits in memoryCommercial OK
2.3BFP16 / BF16≈5.5 GBFits in memoryCommercial OK
1.1BFP16 / BF16≈3.0 GBFits in memoryCommercial OK
212MFP16 / BF16≈1.0 GBFits in memoryNon-commercial
BAAI logobge-m3BAAI
558MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
8.4BQ8_0≈10 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
8.4BQ8_0≈10 GBFits in memoryCommercial OK
Contrastive-LM logoCLM-v0.1-8BContrastive-LM
8BQ8_0≈10 GBFits in memoryCommercial OK
596MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
4BFP16 / BF16≈9.5 GBFits in memoryCommercial OK
2.1BFP16 / BF16≈5.3 GBFits in memoryCommercial OK
8.8BQ8_0≈11 GBFits in memoryCommercial OK
8.2BQ8_0≈10 GBFits in memoryCommercial OK
597MFP16 / BF16≈1.8 GBFits in memoryNon-commercial
568MFP16 / BF16≈1.8 GBFits in memoryCommercial OK
597MFP16 / BF16≈1.8 GBFits in memoryNon-commercial
1.2BFP16 / BF16≈3.3 GBFits in memoryCommercial OK
4BFP16 / BF16≈9.5 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
7.1BQ8_0≈8.9 GBFits in memoryNon-commercial
7.1BQ8_0≈8.9 GBFits in memoryNon-commercial
Tongyi-MAI logoZ-Image-TurboTongyi-MAI
6.2BQ8_0≈7.8 GBFits in memoryCommercial OK
OpenMOSS-Team logoMOSS-Image-SFTOpenMOSS-Team
5.1BQ8_0≈6.5 GBFits in memoryCommercial OK
OpenMOSS-Team logoMOSS-Image-BaseOpenMOSS-Team
5.1BQ8_0≈6.5 GBFits in memoryCommercial OK
Tongyi-MAI logoZ-ImageTongyi-MAI
6.2BQ8_0≈7.8 GBFits in memoryCommercial OK
8.1BQ8_0≈10 GBFits in memoryNon-commercial
6.2BQ8_0≈7.8 GBFits in memoryCommercial OK
speridlabs logoiris-3bsperidlabs
3BFP16 / BF16≈7.2 GBFits in memoryCommercial OK
8BQ8_0≈10 GBFits in memoryCommercial OK
8BQ8_0≈10 GBFits in memoryCommercial OK
9.4BQ4_K_M≈6.9 GBFits in memoryNon-commercial
ModelParamsBest quantMemoryLicence
black-forest-labs logoflux-3-image-klein-comfyblack-forest-labs
5.6BQ8_0≈7.2 GBFits in memoryCheck licence
black-forest-labs logoflux-3-image-kleinblack-forest-labs
5.5BQ8_0≈7.1 GBFits in memoryCheck licence
black-forest-labs logoFLUX.2-klein-9Bblack-forest-labs
9.1BQ8_0≈11 GBFits in memoryNon-commercial
black-forest-labs logoflux-3-image-klein-baseblack-forest-labs
5.5BQ8_0≈7.1 GBFits in memoryCheck licence
black-forest-labs logoFLUX.2-klein-4Bblack-forest-labs
3.9BFP16 / BF16≈9.2 GBFits in memoryCommercial OK
8.1BQ8_0≈10 GBFits in memoryNon-commercial
black-forest-labs logoFLUX.2-klein-base-9Bblack-forest-labs
9.1BQ8_0≈11 GBFits in memoryNon-commercial
8.8BQ8_0≈11 GBFits in memoryCommercial OK
black-forest-labs logoFLUX.2-klein-base-4Bblack-forest-labs
3.9BFP16 / BF16≈9.2 GBFits in memoryCommercial OK
black-forest-labs logoFLUX.2-klein-9b-kvblack-forest-labs
9.1BQ8_0≈11 GBFits in memoryNon-commercial
8.8BQ8_0≈11 GBFits in memoryCommercial OK
6.2BQ8_0≈7.8 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
5BQ8_0≈6.4 GBFits in memoryCommercial OK
8.3BQ8_0≈10 GBFits in memoryConditions
1.4BFP16 / BF16≈3.7 GBFits in memoryCommercial OK
Lightricks logoLTX-VideoLightricks
1.9BFP16 / BF16≈4.8 GBFits in memoryCheck licence
ByteDance-Seed logoSeedVR2-3BByteDance-Seed
3BFP16 / BF16≈7.2 GBFits in memoryCommercial OK
2.2BFP16 / BF16≈5.3 GBFits in memoryCommercial OK
ByteDance-Seed logoSeedVR2-7BByteDance-Seed
7BQ8_0≈8.8 GBFits in memoryCommercial OK
decart-ai logoLucy-Edit-Devdecart-ai
5BQ8_0≈6.4 GBFits in memoryNon-commercial
baidu logoNAVAbaidu
6.3BQ8_0≈8.0 GBFits in memoryCommercial OK
5BQ8_0≈6.4 GBFits in memoryCommercial OK
Efficient-Large-Model logoSANA-Video_2.0_5B_720pEfficient-Large-Model
5BQ8_0≈6.4 GBFits in memoryCommercial OK
1.4BFP16 / BF16≈3.7 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
33MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
155MFP16 / BF16≈0.8 GBFits in memoryCommercial OK
29MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
IDEA-Research logogrounding-dino-tinyIDEA-Research
172MFP16 / BF16≈0.9 GBFits in memoryCommercial OK
29MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
43MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
20MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
68MFP16 / BF16≈0.7 GBFits in memoryCommercial OK
43MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
29MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
31MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
155MFP16 / BF16≈0.8 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
facebook logosam3facebook
860MFP16 / BF16≈2.4 GBFits in memoryCheck licence
ZhengPeng7 logoBiRefNetZhengPeng7
221MFP16 / BF16≈1.0 GBFits in memoryCommercial OK
4MFP16 / BF16≈0.5 GBFits in memoryCheck licence
PramaLLC logoBEN2PramaLLC
95MFP16 / BF16≈0.7 GBFits in memoryCommercial OK
briaai logoRMBG-2.0briaai
221MFP16 / BF16≈1.0 GBFits in memoryCheck licence
632MFP16 / BF16≈1.9 GBFits in memoryCommercial OK
64MFP16 / BF16≈0.6 GBFits in memoryCheck licence
27MFP16 / BF16≈0.6 GBFits in memoryCheck licence
facebook logosam-vit-basefacebook
94MFP16 / BF16≈0.7 GBFits in memoryCommercial OK
ZhengPeng7 logoBiRefNet_liteZhengPeng7
44MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
151MFP16 / BF16≈0.8 GBFits in memoryCommercial OK
39MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
depth-anything logoDA3METRIC-LARGEdepth-anything
334MFP16 / BF16≈1.3 GBFits in memoryCommercial OK
depth-anything logoDA3MONO-LARGEdepth-anything
334MFP16 / BF16≈1.3 GBFits in memoryCommercial OK
1.7BFP16 / BF16≈4.3 GBFits in memoryNon-commercial
depth-anything logoDA3NESTED-GIANT-LARGEdepth-anything
1.7BFP16 / BF16≈4.3 GBFits in memoryNon-commercial
25MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
25MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
952MFP16 / BF16≈2.6 GBFits in memoryNon-commercial
depth-anything logoDA3-GIANTdepth-anything
1.4BFP16 / BF16≈3.5 GBFits in memoryNon-commercial
depth-anything logoDA3-LARGEdepth-anything
411MFP16 / BF16≈1.4 GBFits in memoryNon-commercial
depth-anything logoDA3-LARGE-1.1depth-anything
411MFP16 / BF16≈1.4 GBFits in memoryCommercial OK
depth-anything logoDA3-SMALLdepth-anything
34MFP16 / BF16≈0.6 GBFits in memoryCommercial OK
335MFP16 / BF16≈1.3 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
486MFP16 / BF16≈1.6 GBFits in memoryCommercial OK
303MFP16 / BF16≈1.2 GBFits in memoryConditions
86MFP16 / BF16≈0.7 GBFits in memoryConditions
151MFP16 / BF16≈0.8 GBFits in memoryCheck licence
22MFP16 / BF16≈0.6 GBFits in memoryConditions
375MFP16 / BF16≈1.3 GBFits in memoryCommercial OK
841MFP16 / BF16≈2.4 GBFits in memoryConditions
428MFP16 / BF16≈1.5 GBFits in memoryCheck licence
1.1BFP16 / BF16≈3.0 GBFits in memoryCommercial OK
87MFP16 / BF16≈0.7 GBFits in memoryCommercial OK
203MFP16 / BF16≈0.9 GBFits in memoryCommercial OK
86MFP16 / BF16≈0.7 GBFits in memoryCommercial OK
ModelParamsBest quantMemoryLicence
tencent logoHy-MT2-7Btencent
8BQ8_0≈10 GBFits in memoryCommercial OK
2BFP16 / BF16≈5.1 GBFits in memoryCommercial OK
2.3BFP16 / BF16≈5.6 GBFits in memoryCommercial OK
9.7BQ4_K_M≈7.0 GBFits in memoryCommercial OK
8BQ8_0≈10 GBFits in memoryCheck licence
2BFP16 / BF16≈5.1 GBFits in memoryCheck licence
ByteDance-Seed logoSeed-X-PPO-7BByteDance-Seed
7.5BQ8_0≈9.4 GBFits in memoryCommercial OK
8BQ8_0≈10 GBFits in memoryCheck licence
ByteDance-Seed logoSeed-X-Instruct-7BByteDance-Seed
7.5BQ8_0≈9.4 GBFits in memoryCommercial OK
890MFP16 / BF16≈2.5 GBFits in memoryConditions
4.3BFP16 / BF16≈10 GBFits in memoryCommercial OK
8BQ8_0≈10 GBFits in memoryCheck licence

Similar hardware

Memory figures are estimates: weights at the given quantisation plus an 8K-token KV cache and runtime overhead. Speeds assume memory-bound decoding on a single GPU and will vary with the runtime, drivers and context length.