# h2o-lightning-31b

> H2O-Lightning-31B is a 31B-parameter decision model from H2O.ai, built on google/gemma-4-31B-it. It answers typed decision questions about a record (a document, a ticket, a policy, a conversation), and about images that come with the record. It returns a probability for every option:

- **Organisation:** h2oai
- **Category:** Vision-Language Models
- **Parameters:** 31.3B
- **Licence:** h2o-model-non-commercial-licence-v1.1 (Non-commercial)
- **Released:** 2026-10-20
- **Hugging Face:** 586 downloads (30 days), 8 likes

## Estimated memory

| Quant | ≈ GB |
|---|---|
| Q2_K | 12.58 |
| Q3_K_M | 17.47 |
| Q4_K_M | 21.06 |
| Q5_K_M | 24.27 |
| Q6_K | 27.72 |
| Q8_0 | 35.02 |
| F16 | 63.69 |

## Runs with

- [llama.cpp](https://selfhost.directory/project/llamacpp)
- [SGLang](https://selfhost.directory/project/sglang)
- [Transformers](https://github.com/huggingface/transformers)
- [vLLM](https://selfhost.directory/project/vllm)

## About

H2O-Lightning-31B is a 31B-parameter decision model from H2O.ai, built on google/gemma-4-31B-it. It answers typed decision questions about a record (a document, a ticket, a policy, a conversation), and about images that come with the record. It returns a probability for every option:

• choice: picks one of a set of named options.
• yes/no (noul): gives the probability that a statement is true.
• score: gives an ordinal level on a stated scale, with its probability distribution.

## Alternatives

- [Qwen/Qwen3.8-27B](https://selfhost.directory/models/qwen/qwen3.8-27b) (27.8B, Apache 2.0, ≈18.97 GB at Q4): Following the widespread community adoption of the Qwen3.5 and Qwen3.6 series, we are pleased to introduce Qwen3.8, the most capable generation in the Qwen open-model family to date.
- [google/gemma-4-31B-it](https://selfhost.directory/models/google/gemma-4-31b-it) (31.3B, Apache 2.0, ≈26.54 GB at Q4): Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants.
- [openbmb/MiniCPM-V-4.7-35B-A3B](https://selfhost.directory/models/openbmb/minicpm-v-4.7-35b-a3b) (35.2B (3B active), Apache 2.0, ≈22 GB at Q4): GitHub · On-device results · Evaluation · Examples · Usage · Model weights
- [Qwen/Qwen3.6-35B-A3B](https://selfhost.directory/models/qwen/qwen3.6-35b-a3b) (36B (3B active), Apache 2.0, ≈22.44 GB at Q4): Following the February release of the Qwen3.5 series, we're pleased to share the first open-weight variant of Qwen3.6. Built on direct feedback from the community, Qwen3.6 prioritizes stability and real-world utility, offering developers a more intuitive, responsive, and genuinely productive coding…
- [Qwen/Qwen3.6-27B](https://selfhost.directory/models/qwen/qwen3.6-27b) (27.8B, Apache 2.0, ≈18.97 GB at Q4): Following the February release of the Qwen3.5 series, we're pleased to share the first open-weight variant of Qwen3.6. Built on direct feedback from the community, Qwen3.6 prioritizes stability and real-world utility, offering developers a more intuitive, responsive, and genuinely productive coding…
- [google/gemma-4-26B-A4B-it](https://selfhost.directory/models/google/gemma-4-26b-a4b-it) (25.8B (4B active), Apache 2.0, ≈17.67 GB at Q4): Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants.

More: https://selfhost.directory/models/h2oai/h2o-lightning-31b/alternatives

## Runs well on

- [Tesla P40](https://selfhost.directory/models/h2oai/h2o-lightning-31b/on/tesla-p40): Q4_K_M, est. 10 tok/s
- [Radeon RX 7900 XTX](https://selfhost.directory/models/h2oai/h2o-lightning-31b/on/rx-7900-xtx): Q4_K_M, est. 28 tok/s
- [Instinct MI50 32GB](https://selfhost.directory/models/h2oai/h2o-lightning-31b/on/instinct-mi50): Q6_K, est. 22 tok/s
- [GeForce RTX 5090](https://selfhost.directory/models/h2oai/h2o-lightning-31b/on/rtx-5090): Q6_K, est. 38 tok/s
- [Apple M4 Pro (48 GB)](https://selfhost.directory/models/h2oai/h2o-lightning-31b/on/apple-m4-pro-48gb): Q6_K, est. 5.8 tok/s
- [RTX A6000](https://selfhost.directory/models/h2oai/h2o-lightning-31b/on/rtx-a6000): Q8_0, est. 13 tok/s

## Similar size

- [Qwen/Qwen3.8-27B](https://selfhost.directory/models/qwen/qwen3.8-27b) (27.8B)
- [google/gemma-4-31B-it](https://selfhost.directory/models/google/gemma-4-31b-it) (31.3B)
- [Qwen/Qwen3.6-35B-A3B](https://selfhost.directory/models/qwen/qwen3.6-35b-a3b) (36B)
- [google/gemma-4-26B-A4B-it](https://selfhost.directory/models/google/gemma-4-26b-a4b-it) (25.8B)
- [Qwen/Qwen3.6-27B](https://selfhost.directory/models/qwen/qwen3.6-27b) (27.8B)
- [Qwen/Qwen3.5-35B-A3B](https://selfhost.directory/models/qwen/qwen3.5-35b-a3b) (36B)

## FAQ

**How much VRAM does h2o-lightning-31b need?**
About 21 GB at Q4_K_M with an 8K context. Q8_0 needs about 35 GB and full precision about 64 GB. Longer contexts need more memory for the KV cache.

**Can h2o-lightning-31b run on the Tesla P40?**
Yes. It fits at Q4_K_M (about 21 GB), at an estimated 10 tokens/s. It fits comfortably on the Radeon RX 7900 XTX, Instinct MI50 32GB, RTX 5090.

**Can I use h2o-lightning-31b commercially?**
No. It is released under h2o-model-non-commercial-licence-v1.1. Custom licence: read the model card before using it commercially.

**How do I run h2o-lightning-31b locally?**
With vLLM: vllm serve h2oai/h2o-lightning-31b.

## Links

- huggingface: https://huggingface.co/h2oai/h2o-lightning-31b
- github: https://github.com/h2oai/vllm
- demo: https://huggingface.co/spaces/multimodalart/jev-decision-index

---
Source: https://selfhost.directory/models/h2oai/h2o-lightning-31b