selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

tencent logo

Can the Arc A580 run WeVisCut?

WeVisCut·9.4B·Arc A580·8 GB VRAM

Saved in this browser, used on every model page.
Yes

Yes. The Arc A580 runs WeVisCut fully in VRAM at Q4_K_M (about 7.0 GB of 8.0 GB), at roughly 49 tokens/s.

best quant
Q4_K_M
needs
7.0 GB
usable
8.0 GB
est. speed
~49 tok/s

Estimates for an 8K context; offload assumes 32 GB of system RAM.Try other contexts on the WeVisCut page →

Every quantisation of WeVisCut on the Arc A580

QuantQualityMemoryFitSpeed
Q2_K Heavy loss4.5 GBfits~91 tok/s
Q3_K_M Noticeable loss6.0 GBfits~61 tok/s
Q4_K_M pickSmall loss7.0 GBfits~49 tok/s
Q5_K_M Near-lossless7.8 GBtight~42 tok/s
Q6_K Near-lossless8.7 GBoffload~23 tok/s
Q8_0 Lossless11 GBoffload~9.5 tok/s
FP16 / BF16 Lossless19 GBoffload~2.8 tok/s

Speed is a bandwidth-based estimate for generating text (512 GB/s); prompt processing and long contexts are slower.

Measured speed on the Arc A580

No community reports for this pair yet.

Report your own speed →

Run WeVisCut on the Arc A580

Using Q4_K_M, the best fit for Arc A580.

llama.cpp · OpenAI-compatible server with a web UIdocs
llama-server -hf prithivMLmods/tencent-WeVisCut-GGUF:Q4_K_M

Needs a Vulkan build of llama.cpp. Downloads the GGUF on first run.

llama.cpp · Docker with Vulkandocs
docker run -d --name llama --device /dev/dri -p 8080:8080 \
  -v ~/.cache/llama.cpp:/root/.cache/llama.cpp \
  ghcr.io/ggml-org/llama.cpp:server-vulkan \
  -hf prithivMLmods/tencent-WeVisCut-GGUF:Q4_K_M --host 0.0.0.0 --port 8080 -ngl 99

Server and web UI on port 8080.

Omni models that fit the Arc A580 comfortably

All that fit →
ModelParamsBest quantMemorySpeedLicence
9.4BQ5_K_M≈7.3 GBFits in memory~42 tok/sCommercial OK
8BQ5_K_M≈6.5 GBFits in memory~50 tok/sCommercial OK
10.7BQ6_K≈7.0 GBFits in memory~32 tok/sCommercial OK
9.1BQ4_K_M≈7.0 GBFits in memory~51 tok/sCommercial OK
8.3BQ6_K≈7.6 GBFits in memory~41 tok/sCheck licence
8.7BQ6_K≈7.0 GBFits in memory~39 tok/sCommercial OK

Other GPUs that run WeVisCut

Frequently asked questions

Can the Arc A580 run WeVisCut?
Yes. The Arc A580 runs WeVisCut fully in VRAM at Q4_K_M (about 7.0 GB of 8.0 GB), at roughly 49 tokens/s.
How fast does WeVisCut run on the Arc A580?
An estimate from memory bandwidth gives about 49 tokens/s at Q4_K_M.
Which quantisation of WeVisCut should I use on the Arc A580?
Q4_K_M: the largest one that fits. Expected quality: small loss compared with the full-precision weights.

Memory and speed are estimates from the model's size and the card's memory bandwidth; real numbers depend on the runtime, drivers and context length. Community figures are medians of user reports.