selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

ATH-MaaS logo

Can the Tesla P40 run OvisOCR3?

OvisOCR3·853M·Tesla P40·24 GB VRAM

Saved in this browser, used on every model page.
Yes

Yes. The Tesla P40 runs OvisOCR3 fully in VRAM at FP16 / BF16 (about 2.2 GB of 24 GB).

best quant
FP16 / BF16
needs
2.2 GB
usable
24 GB

Estimates from the model's size.Try other contexts on the OvisOCR3 page →

Every quantisation of OvisOCR3 on the Tesla P40

QuantMemoryFit
Q4_K_M 1.1 GBfits
Q8_0 1.4 GBfits
FP16 / BF16 pick2.2 GBfits

Speed is a bandwidth-based estimate for generating text (347 GB/s); prompt processing and long contexts are slower.

Run OvisOCR3 on the Tesla P40

Using FP16 / BF16, the best fit for Tesla P40.

Ollama · F16 straight from Hugging Facedocs
ollama run hf.co/mradermacher/OvisOCR3-GGUF:F16
llama.cpp · OpenAI-compatible server with a web UIdocs
llama-server -hf mradermacher/OvisOCR3-GGUF:F16

Downloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.

LM Studio · Download with the lms CLIdocs
lms get mradermacher/OvisOCR3-GGUF@F16

Then load it from the LM Studio app or with `lms load`.

vLLM · Serve an OpenAI-compatible APIdocs
vllm serve ATH-MaaS/OvisOCR3
vLLM · Smaller AWQ builddocs
vllm serve leleroyn/OvisOCR3-AWQ-W4A16
vLLM · Docker Compose (NVIDIA)docs
services:
  vllm:
    image: vllm/vllm-openai:latest
    command: ATH-MaaS/OvisOCR3
    ports:
      - "8000:8000"
    ipc: host
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

API on port 8000.

SGLang · Serve an OpenAI-compatible APIdocs
sglang serve ATH-MaaS/OvisOCR3 --host 0.0.0.0 --port 30000
SGLang · Docker Compose (NVIDIA)docs
services:
  sglang:
    image: lmsysorg/sglang:latest
    command: sglang serve ATH-MaaS/OvisOCR3 --host 0.0.0.0 --port 30000
    ports:
      - "30000:30000"
    ipc: host
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

API on port 30000.

Transformers · Ask about an imagedocs
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ATH-MaaS/OvisOCR3", dtype="auto", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg"},
            {"type": "text", "text": "Describe this image."},
        ],
    }
]
out = pipe(text=messages, max_new_tokens=256)
print(out[0]["generated_text"][-1]["content"])

OCR models that fit the Tesla P40 comfortably

All that fit →
ModelParamsBest quantMemoryLicence
PaddlePaddle logoPaddleOCR-VL-1.6PaddlePaddle
959MFP16 / BF16≈2.6 GBFits in memoryCommercial OK
PaddlePaddle logoPaddleOCR-VLPaddlePaddle
959MFP16 / BF16≈2.6 GBFits in memoryCommercial OK
tencent logoHunyuanOCRtencent
1.1BFP16 / BF16≈3.0 GBFits in memoryConditions
1BFP16 / BF16≈2.8 GBFits in memoryCommercial OK
853MFP16 / BF16≈2.4 GBFits in memoryCommercial OK
datalab-to logosurya-ocr-2.5datalab-to
689MFP16 / BF16≈2.0 GBFits in memoryConditions

Other GPUs that run OvisOCR3

Frequently asked questions

Can the Tesla P40 run OvisOCR3?
Yes. The Tesla P40 runs OvisOCR3 fully in VRAM at FP16 / BF16 (about 2.2 GB of 24 GB).
Which quantisation of OvisOCR3 should I use on the Tesla P40?
FP16 / BF16: the largest one that fits. Expected quality: lossless compared with the full-precision weights.

Memory and speed are estimates from the model's size and the card's memory bandwidth; real numbers depend on the runtime, drivers and context length. Community figures are medians of user reports.