selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

GroundingPI logo

GroundAnything-VLM

GroundingPI·Instruct·released 30 Sept 2026

apache-2.0-with-upstream-kimi-k34.8B256K ctx ≈4.5 GB
Your GPU
hf download GroundingPI/GroundAnything-VLM
Setup ↓

Model family: GroundAnything — DLM / parallel decoding · GroundAnything-VLM — autoregressive.

Notify me when

How much VRAM does GroundAnything-VLM need?

At Q4_K_M GroundAnything-VLM needs about 4.5 GB with an 8K context (4.5 GB at Q4, 11 GB at full precision). Pick your GPU to see what fits and how fast it runs.

--vram
any
--gpus
--quant

Needs: ≈4.3 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.

Change quant, context or KV cache to compare
Advanced: context 8K · KV cache 16-bit
Context
KV cache

Smallest GPU for each quant and context (KV cache 16-bit)

Quantisation8K16K32K64K128K256K
Q3_K_M
Q4_K_M
Q5_K_M
Q6_K
Q8_0
FP16

≤16 GB card24–32 GB48–96 GBmore

QuantisationQualityWeightsKV cacheTotal
Q2_KHeavy lossSmallest, noticeably worse qualityHeavy loss1.5 GB est.1.1 GB≈3.2 GB
Q3_K_MNoticeable lossSmall, some quality lossNoticeable loss2.2 GB est.1.1 GB≈3.9 GB
Q4_K_MrecommendedSmall lossRecommended default for local useSmall loss2.5 GB file1.1 GB≈4.3 GB
Q5_K_MNear-losslessNear-lossless for most tasksNear-lossless3.2 GB est.1.1 GB≈5.0 GB
Q6_KNear-losslessPractically losslessNear-lossless3.7 GB est.1.1 GB≈5.5 GB
Q8_0LosslessLossless in practice, twice the size of Q4Lossless4.8 GB est.1.1 GB≈6.7 GB
FP16 / BF16LosslessFull precision, as releasedLossless9.0 GB est.1.1 GB≈11 GB

Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it. Weights marked “file” use the real GGUF download size; “est.” rows are computed from the parameter count.

How to pick a quant

Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.

  • LosslessF16, Q8_0
  • Near-losslessQ6_K, Q5_K_M
  • Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
  • Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
  • Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K

Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B

How much context fits?

Model quant
KV cache

At 8K context, Q4_K_M needs ≈4.3 GB(2.5 GB weights + 1.1 GB KV cache + 0.6 GB overhead). Pick a GPU above to see how much context fits.

010203040501K2K4K8K16K32K64K128K256KGB

Weights + overheadKV cache

Can the RTX 4060 run GroundAnything-VLM? →

Speed vs quality per quant

Smaller quants run faster and fit in less memory, but lose quality.

Q2_KQ3_K_MQ4_K_MQ5_K_MQ6_KQ8_0F16

Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.

Real-world speed of GroundAnything-VLM

Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →

No medians yet. Be one of the first 3 to report a setup.

All reports

Run GroundAnything-VLM locally

These commands load the original weights from Hugging Face.

Ask about an imagedocs
python
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="GroundingPI/GroundAnything-VLM", dtype="auto", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"},
            {"type": "text", "text": "Describe this image."},
        ],
    }
]
out = pipe(text=messages, max_new_tokens=256)
print(out[0]["generated_text"][-1]["content"])

Install Transformers

Quantised versions and downloads

RepositoryFormatQuantSizeDownloads
woddle/GroundAnything-VLM-Q4_K_M-GGUFGGUFQ4_K_M2.7 GB275

About GroundAnything-VLM

Model family: GroundAnything — DLM / parallel decoding · GroundAnything-VLM — autoregressive. This repository contains the autoregressive GroundAnything-VLM checkpoint. The family overview and figures below are shared with the DLM page; use the VLM serving recipe for these weights. • ModelScope: GroundAnything (DLM) · GroundAnything-VLM (VLM). • 🚀 Online Demo: Hugging Face Spaces. • 🌐 Project Page: GroundAnything. • 💻 GitHub Code: groundingpi/GroundAnything. • 📄 Paper: arXiv:2609.39600. • 🤗 HF paper: Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising.

Full model card on Hugging Face →

Alternatives to GroundAnything-VLM

All alternatives →
Qwen logo

Qwen3.5-4B

Qwen·Vision

≈4.3 GB

Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance.

4.7B256K ctx Apache 2.0
4.3MFeb 2026ollama
Qwen logo≈4.3 GB

Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date.

4.4B256K ctx Apache 2.0
2.8MOct 2025ollama
Qwen logo

Qwen3.5-4B-Base

Qwen·Vision

≈4.3 GB

Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance.

4.7B256K ctx Apache 2.0
496.7KFeb 2026gguf
tencent logo

WeVisOne-VL-4B

tencent·Vision

≈4.5 GB

WeVisOne-VL (WeChat Vision All-in-One) is a unified vision-language model developed by WeChat Vision, Tencent Inc. Built upon Qwen3-VL, it unifies object-level perception (open-vocabulary object detection, referring localization, object counting, object pointing, and OCR) and pixel-level perception…

4.8B256K ctx Apache 2.0
36.4KOct 2026gguf
h2oai logo

h2o-lightning-4b

h2oai·Vision

≈4.2 GB

H2O-Lightning-4B is a 4B-parameter decision model from H2O.ai, built on Qwen/Qwen3.5-4B. It answers typed decision questions about a record (a document, a ticket, a policy, a conversation) and about images that come with the record (photos, screenshots, scanned documents, charts).

4.5B256K ctx Apache 2.0
26.1KOct 2026gguf
google logo

medgemma-1.5-4b-it

google·Vision

≈4.1 GB

* License: The use of MedGemma is governed by the Health AI Developer Foundations terms of use. MedGemma has not been evaluated or optimized for multi-turn applications.

4.3B128K ctx health-ai-developer-foundations
183.5KJan 2026ollama

GroundAnything-VLM FAQ

How much VRAM does GroundAnything-VLM need?
About 4.5 GB at Q4_K_M with an 8K context. Q8_0 needs about 6.7 GB and full precision about 11 GB. Longer contexts need more memory for the KV cache.
Can GroundAnything-VLM run on the RTX 4060?
Yes. It fits at Q8_0 (about 6.7 GB), at an estimated 29 tokens/s. It fits comfortably on the Apple M4 (16 GB), Radeon RX 6700 XT, RTX 3060 12GB.
Can I use GroundAnything-VLM commercially?
Yes. It is released under apache-2.0-with-upstream-kimi-k3. Custom licence: read the model card before using it commercially.
What is the context length of GroundAnything-VLM?
262,144 tokens (256K). The full window needs far more memory than an 8K context.
How do I run GroundAnything-VLM locally?
It runs with Transformers. Each has copy-paste examples in its docs.

Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.