selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

tencent logo

Can the RTX 3060 12GB run WeVisCut?

WeVisCut·9.4B·GeForce RTX 3060 12GB·12 GB VRAM

Saved in this browser, used on every model page.
Yes

Yes. The RTX 3060 12GB runs WeVisCut fully in VRAM at Q8_0 (about 11 GB of 12 GB), at roughly 20 tokens/s.

best quant
Q8_0
needs
11 GB
usable
12 GB
est. speed
~20 tok/s

Estimates for an 8K context; offload assumes 32 GB of system RAM.Try other contexts on the WeVisCut page →

Every quantisation of WeVisCut on the RTX 3060 12GB

QuantQualityMemoryFitSpeed
Q2_K Heavy loss4.5 GBfits~64 tok/s
Q3_K_M Noticeable loss6.0 GBfits~43 tok/s
Q4_K_M Small loss7.0 GBfits~35 tok/s
Q5_K_M Near-lossless7.8 GBfits~30 tok/s
Q6_K Near-lossless8.7 GBfits~26 tok/s
Q8_0 pickLossless11 GBfits~20 tok/s
FP16 / BF16 Lossless19 GBoffload~3.7 tok/s

Speed is a bandwidth-based estimate for generating text (360 GB/s); prompt processing and long contexts are slower.

Measured speed on the RTX 3060 12GB

No community reports for this pair yet.

Report your own speed →

Run WeVisCut on the RTX 3060 12GB

Using Q8_0, the best fit for RTX 3060 12GB.

llama.cpp · OpenAI-compatible server with a web UIdocs
llama-server -hf prithivMLmods/tencent-WeVisCut-GGUF:Q8_0

Downloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.

vLLM · Serve an OpenAI-compatible APIdocs
vllm serve tencent/WeVisCut
vLLM · Docker Compose (NVIDIA)docs
services:
  vllm:
    image: vllm/vllm-openai:latest
    command: tencent/WeVisCut
    ports:
      - "8000:8000"
    ipc: host
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

API on port 8000.

SGLang · Serve an OpenAI-compatible APIdocs
sglang serve tencent/WeVisCut --host 0.0.0.0 --port 30000
SGLang · Docker Compose (NVIDIA)docs
services:
  sglang:
    image: lmsysorg/sglang:latest
    command: sglang serve tencent/WeVisCut --host 0.0.0.0 --port 30000
    ports:
      - "30000:30000"
    ipc: host
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

API on port 30000.

Omni models that fit the RTX 3060 12GB comfortably

All that fit →
ModelParamsBest quantMemorySpeedLicence
9.4BQ8_0≈10 GBFits in memory~20 tok/sCommercial OK
8BQ8_0≈9.2 GBFits in memory~23 tok/sCommercial OK
10.7BQ8_0≈8.9 GBFits in memory~17 tok/sCommercial OK
9.1BQ8_0≈10 GBFits in memory~21 tok/sCommercial OK
12BQ4_K_M≈10 GBFits in memory~27 tok/sCommercial OK
8.3BQ8_0≈10 GBFits in memory~23 tok/sCheck licence

Other GPUs that run WeVisCut

Frequently asked questions

Can the RTX 3060 12GB run WeVisCut?
Yes. The RTX 3060 12GB runs WeVisCut fully in VRAM at Q8_0 (about 11 GB of 12 GB), at roughly 20 tokens/s.
How fast does WeVisCut run on the RTX 3060 12GB?
An estimate from memory bandwidth gives about 20 tokens/s at Q8_0.
Which quantisation of WeVisCut should I use on the RTX 3060 12GB?
Q8_0: the largest one that fits. Expected quality: lossless compared with the full-precision weights.

Memory and speed are estimates from the model's size and the card's memory bandwidth; real numbers depend on the runtime, drivers and context length. Community figures are medians of user reports.