AI models that run on the GeForce RTX 3060 12GB
With 12 GB of VRAM, the GeForce RTX 3060 12GB can run 2 343 open-weight models fully in memory at their recommended quantisation. Each section puts the strongest models that fit comfortably first, with the best quantisation that still fits, and an estimated generation speed.
The GeForce RTX 3060 12GB has 12 GB of GDDR6 memory, and all 12 GB is usable for models. Its 360 GB/s memory bandwidth affects how quickly a model can produce output, so memory size determines which models can load and bandwidth determines how fast they respond. The card supports CUDA and Vulkan backends. Popular open-weight models that fit include gemma-4-E4B-it, Llama-3.1-8B-Instruct, and Qwen3-8B.
- VRAM
- 12 GB
- usable for models
- ≈12 GB
- bandwidth
- 360 GB/s
- backends
- cuda, vulkan
- memory
- GDDR6 · 192-bit
- power
- 170 W
- vendor · year
- NVIDIA · 2021
- price · value ranking →
- $329launch MSRP, 2021
Specs: Wikipedia (CC BY-SA 4.0)
| Model | Params | Best quant | Memory | Speed |
|---|---|---|---|---|
| 8.2B | Q8_0 | ≈10 GBFits in memory | ~23 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 9.7B | Q8_0 | ≈11 GBFits in memory | ~19 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 7.9BA1.4B | Q8_0 | ≈10 GBFits in memory | ~135 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 9B | Q8_0 | ≈11 GBFits in memory | ~21 tok/s | |
| 14.7B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 8.8B | Q8_0 | ≈11 GBFits in memory | ~21 tok/s | |
| 9.2B | Q6_K | ≈11 GBFits in memory | ~26 tok/s | |
| 3.7B | Q8_0 | ≈4.9 GBFits in memory | ~51 tok/s | |
| 7.6B | Q8_0 | ≈8.9 GBFits in memory | ~24 tok/s |
| Model | Params | Best quant | Memory | Speed |
|---|---|---|---|---|
| 12.1BA2.5B | Q5_K_M | ≈11 GBFits in memory | ~111 tok/s | |
| 12.1BA2.5B | Q6_K | ≈11 GBFits in memory | ~96 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 7.6B | Q8_0 | ≈8.9 GBFits in memory | ~24 tok/s | |
| 16B | Q4_K_M | ≈11 GBFits in memory | ~20 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 4B | Q8_0 | ≈7.5 GBFits in memory | ~46 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 8.3B | Q8_0 | ≈10 GBFits in memory | ~23 tok/s | |
| 8.5B | Q6_K | ≈11 GBFits in memory | ~28 tok/s |
| Model | Params | Best quant | Memory | Speed |
|---|---|---|---|---|
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 14.7B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 14.7B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 7.3B | Q6_K | ≈10 GBFits in memory | ~33 tok/s | |
| 8.6B | Q8_0 | ≈10 GBFits in memory | ~22 tok/s | |
| 14.8B | Q4_K_M | ≈11 GBFits in memory | ~22 tok/s | |
| 16.3BA1.4B | Q4_K_M | ≈11 GBFits in memory | ~228 tok/s | |
| 14.7B | Q4_K_M | ≈10 GBFits in memory | ~22 tok/s | |
| 7.6B | Q8_0 | ≈8.9 GBFits in memory | ~24 tok/s | |
| 8.2B | Q8_0 | ≈10 GBFits in memory | ~23 tok/s | |
| 7.3B | Q6_K | ≈10 GBFits in memory | ~33 tok/s | |
| 8B | Q8_0 | ≈9.8 GBFits in memory | ~23 tok/s |
| Model | Params | Best quant | Memory | Speed |
|---|---|---|---|---|
| 9.7B | Q8_0 | ≈11 GBFits in memory | ~19 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 8.8B | Q8_0 | ≈10 GBFits in memory | ~21 tok/s | |
| 9.7B | Q8_0 | ≈11 GBFits in memory | ~19 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 4.7B | Q8_0 | ≈5.9 GBFits in memory | ~40 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 12.2B | Q4_K_M | ≈10 GBFits in memory | ~27 tok/s | |
| 9.8B | Q8_0 | ≈11 GBFits in memory | ~19 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 8.8B | Q8_0 | ≈10 GBFits in memory | ~21 tok/s | |
| 9B | Q8_0 | ≈11 GBFits in memory | ~21 tok/s |
| Model | Params | Best quant | Memory | Speed |
|---|---|---|---|---|
| 12B | Q4_K_M | ≈10 GBFits in memory | ~27 tok/s | |
| 8B | Q8_0 | ≈9.2 GBFits in memory | ~23 tok/s | |
| 9.4B | Q8_0 | ≈10 GBFits in memory | ~20 tok/s | |
| 12B | Q4_K_M | ≈11 GBFits in memory | ~27 tok/s | |
| 9.4B | Q8_0 | ≈11 GBFits in memory | ~20 tok/s | |
| 9.1B | Q8_0 | ≈10 GBFits in memory | ~21 tok/s | |
| 5.1B | Q8_0 | ≈5.7 GBFits in memory | ~36 tok/s | |
| 8.7B | Q8_0 | ≈8.9 GBFits in memory | ~21 tok/s | |
| 16.3BA1.5B | Q4_K_M | ≈10 GBFits in memory | ~218 tok/s | |
| 7.8B | Q8_0 | ≈8.2 GBFits in memory | ~24 tok/s | |
| 14.7B | Q5_K_M | ≈11 GBFits in memory | ~19 tok/s | |
| 2.2B | Q8_0 | ≈2.7 GBFits in memory | ~86 tok/s |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 3.3BA1.1B | FP16 / BF16 | ≈8.0 GBFits in memory | |
| 4.5B | FP16 / BF16 | ≈11 GBFits in memory | |
| 5.3B | Q8_0 | ≈6.8 GBFits in memory | |
| 1.3B | FP16 / BF16 | ≈3.5 GBFits in memory | |
| 3.3BA1.1B | FP16 / BF16 | ≈8.0 GBFits in memory | |
| 959M | FP16 / BF16 | ≈2.6 GBFits in memory | |
| 853M | FP16 / BF16 | ≈2.4 GBFits in memory | |
| 1.4B | FP16 / BF16 | ≈3.7 GBFits in memory | |
| 1.3B | FP16 / BF16 | ≈3.4 GBFits in memory | |
| 1.1B | FP16 / BF16 | ≈3.0 GBFits in memory | |
| 2.2B | FP16 / BF16 | ≈5.5 GBFits in memory | |
| 1.2B | FP16 / BF16 | ≈3.1 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 2.1B | FP16 / BF16 | ≈5.1 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.8 GBFits in memory | |
| 8.7B | Q8_0 | ≈11 GBFits in memory | |
| 8.7B | Q8_0 | ≈11 GBFits in memory | |
| 4.4B | FP16 / BF16 | ≈10 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.7 GBFits in memory | |
| 627M | FP16 / BF16 | ≈1.9 GBFits in memory | |
| 909M | FP16 / BF16 | ≈2.5 GBFits in memory | |
| 2B | FP16 / BF16 | ≈5.1 GBFits in memory | |
| 938M | FP16 / BF16 | ≈2.6 GBFits in memory | |
| 2.1B | FP16 / BF16 | ≈5.1 GBFits in memory | |
| 600M | FP16 / BF16 | ≈1.8 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 3.6B | FP16 / BF16 | ≈8.6 GBFits in memory | |
| 3.5B | FP16 / BF16 | ≈8.3 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.6 GBFits in memory | |
| 1.9B | FP16 / BF16 | ≈4.8 GBFits in memory | |
| 613M | FP16 / BF16 | ≈1.9 GBFits in memory | |
| 1.5B | FP16 / BF16 | ≈3.8 GBFits in memory | |
| 915M | FP16 / BF16 | ≈2.5 GBFits in memory | |
| 4.6B | FP16 / BF16 | ≈11 GBFits in memory | |
| 8.9BA2.1B | Q8_0 | ≈11 GBFits in memory | |
| 82M | FP16 / BF16 | ≈0.7 GBFits in memory | |
| 601M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 8.5B | Q8_0 | ≈11 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 2.4B | FP16 / BF16 | ≈5.9 GBFits in memory | |
| 8.4B | Q8_0 | ≈10 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.7 GBFits in memory | |
| 5B | Q8_0 | ≈6.4 GBFits in memory | |
| 1.5B | FP16 / BF16 | ≈3.8 GBFits in memory | |
| 1.5B | FP16 / BF16 | ≈3.8 GBFits in memory | |
| 568M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 5B | Q8_0 | ≈6.4 GBFits in memory | |
| 568M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 1.4B | FP16 / BF16 | ≈3.7 GBFits in memory | |
| 171M | FP16 / BF16 | ≈0.9 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.7 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 744M | FP16 / BF16 | ≈2.2 GBFits in memory | |
| 7.6B | Q8_0 | ≈9.5 GBFits in memory | |
| 303M | FP16 / BF16 | ≈1.2 GBFits in memory | |
| 596M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 4B | FP16 / BF16 | ≈9.5 GBFits in memory | |
| 8B | Q8_0 | ≈9.9 GBFits in memory | |
| 8.1B | Q8_0 | ≈10 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.5 GBFits in memory | |
| 1.1B | FP16 / BF16 | ≈3.0 GBFits in memory | |
| 212M | FP16 / BF16 | ≈1.0 GBFits in memory | |
| 558M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 8.4B | Q8_0 | ≈10 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 8.4B | Q8_0 | ≈10 GBFits in memory | |
| 8B | Q8_0 | ≈10 GBFits in memory | |
| 596M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 4B | FP16 / BF16 | ≈9.5 GBFits in memory | |
| 2.1B | FP16 / BF16 | ≈5.3 GBFits in memory | |
| 8.8B | Q8_0 | ≈11 GBFits in memory | |
| 8.2B | Q8_0 | ≈10 GBFits in memory | |
| 597M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 568M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 597M | FP16 / BF16 | ≈1.8 GBFits in memory | |
| 1.2B | FP16 / BF16 | ≈3.3 GBFits in memory | |
| 4B | FP16 / BF16 | ≈9.5 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 7.1B | Q8_0 | ≈8.9 GBFits in memory | |
| 7.1B | Q8_0 | ≈8.9 GBFits in memory | |
| 6.2B | Q8_0 | ≈7.8 GBFits in memory | |
| 5.1B | Q8_0 | ≈6.5 GBFits in memory | |
| 5.1B | Q8_0 | ≈6.5 GBFits in memory | |
| 6.2B | Q8_0 | ≈7.8 GBFits in memory | |
| 8.1B | Q8_0 | ≈10 GBFits in memory | |
| 6.2B | Q8_0 | ≈7.8 GBFits in memory | |
| 3B | FP16 / BF16 | ≈7.2 GBFits in memory | |
| 8B | Q8_0 | ≈10 GBFits in memory | |
| 8B | Q8_0 | ≈10 GBFits in memory | |
| 9.4B | Q4_K_M | ≈6.9 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 5.6B | Q8_0 | ≈7.2 GBFits in memory | |
| 5.5B | Q8_0 | ≈7.1 GBFits in memory | |
| 9.1B | Q8_0 | ≈11 GBFits in memory | |
| 5.5B | Q8_0 | ≈7.1 GBFits in memory | |
| 3.9B | FP16 / BF16 | ≈9.2 GBFits in memory | |
| 8.1B | Q8_0 | ≈10 GBFits in memory | |
| 9.1B | Q8_0 | ≈11 GBFits in memory | |
| 8.8B | Q8_0 | ≈11 GBFits in memory | |
| 3.9B | FP16 / BF16 | ≈9.2 GBFits in memory | |
| 9.1B | Q8_0 | ≈11 GBFits in memory | |
| 8.8B | Q8_0 | ≈11 GBFits in memory | |
| 6.2B | Q8_0 | ≈7.8 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 5B | Q8_0 | ≈6.4 GBFits in memory | |
| 8.3B | Q8_0 | ≈10 GBFits in memory | |
| 1.4B | FP16 / BF16 | ≈3.7 GBFits in memory | |
| 1.9B | FP16 / BF16 | ≈4.8 GBFits in memory | |
| 3B | FP16 / BF16 | ≈7.2 GBFits in memory | |
| 2.2B | FP16 / BF16 | ≈5.3 GBFits in memory | |
| 7B | Q8_0 | ≈8.8 GBFits in memory | |
| 5B | Q8_0 | ≈6.4 GBFits in memory | |
| 6.3B | Q8_0 | ≈8.0 GBFits in memory | |
| 5B | Q8_0 | ≈6.4 GBFits in memory | |
| 5B | Q8_0 | ≈6.4 GBFits in memory | |
| 1.4B | FP16 / BF16 | ≈3.7 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 33M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 155M | FP16 / BF16 | ≈0.8 GBFits in memory | |
| 29M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 172M | FP16 / BF16 | ≈0.9 GBFits in memory | |
| 29M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 43M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 20M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 68M | FP16 / BF16 | ≈0.7 GBFits in memory | |
| 43M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 29M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 31M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 155M | FP16 / BF16 | ≈0.8 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 860M | FP16 / BF16 | ≈2.4 GBFits in memory | |
| 221M | FP16 / BF16 | ≈1.0 GBFits in memory | |
| 4M | FP16 / BF16 | ≈0.5 GBFits in memory | |
| 95M | FP16 / BF16 | ≈0.7 GBFits in memory | |
| 221M | FP16 / BF16 | ≈1.0 GBFits in memory | |
| 632M | FP16 / BF16 | ≈1.9 GBFits in memory | |
| 64M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 27M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 94M | FP16 / BF16 | ≈0.7 GBFits in memory | |
| 44M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 151M | FP16 / BF16 | ≈0.8 GBFits in memory | |
| 39M | FP16 / BF16 | ≈0.6 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 334M | FP16 / BF16 | ≈1.3 GBFits in memory | |
| 334M | FP16 / BF16 | ≈1.3 GBFits in memory | |
| 1.7B | FP16 / BF16 | ≈4.3 GBFits in memory | |
| 1.7B | FP16 / BF16 | ≈4.3 GBFits in memory | |
| 25M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 25M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 952M | FP16 / BF16 | ≈2.6 GBFits in memory | |
| 1.4B | FP16 / BF16 | ≈3.5 GBFits in memory | |
| 411M | FP16 / BF16 | ≈1.4 GBFits in memory | |
| 411M | FP16 / BF16 | ≈1.4 GBFits in memory | |
| 34M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 335M | FP16 / BF16 | ≈1.3 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 486M | FP16 / BF16 | ≈1.6 GBFits in memory | |
| 303M | FP16 / BF16 | ≈1.2 GBFits in memory | |
| 86M | FP16 / BF16 | ≈0.7 GBFits in memory | |
| 151M | FP16 / BF16 | ≈0.8 GBFits in memory | |
| 22M | FP16 / BF16 | ≈0.6 GBFits in memory | |
| 375M | FP16 / BF16 | ≈1.3 GBFits in memory | |
| 841M | FP16 / BF16 | ≈2.4 GBFits in memory | |
| 428M | FP16 / BF16 | ≈1.5 GBFits in memory | |
| 1.1B | FP16 / BF16 | ≈3.0 GBFits in memory | |
| 87M | FP16 / BF16 | ≈0.7 GBFits in memory | |
| 203M | FP16 / BF16 | ≈0.9 GBFits in memory | |
| 86M | FP16 / BF16 | ≈0.7 GBFits in memory |
| Model | Params | Best quant | Memory |
|---|---|---|---|
| 8B | Q8_0 | ≈10 GBFits in memory | |
| 2B | FP16 / BF16 | ≈5.1 GBFits in memory | |
| 2.3B | FP16 / BF16 | ≈5.6 GBFits in memory | |
| 9.7B | Q4_K_M | ≈7.0 GBFits in memory | |
| 8B | Q8_0 | ≈10 GBFits in memory | |
| 2B | FP16 / BF16 | ≈5.1 GBFits in memory | |
| 7.5B | Q8_0 | ≈9.4 GBFits in memory | |
| 8B | Q8_0 | ≈10 GBFits in memory | |
| 7.5B | Q8_0 | ≈9.4 GBFits in memory | |
| 890M | FP16 / BF16 | ≈2.5 GBFits in memory | |
| 4.3B | FP16 / BF16 | ≈10 GBFits in memory | |
| 8B | Q8_0 | ≈10 GBFits in memory |
Similar hardware
Memory figures are estimates: weights at the given quantisation plus an 8K-token KV cache and runtime overhead. Speeds assume memory-bound decoding on a single GPU and will vary with the runtime, drivers and context length.