We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.
LFM3-1.7B
LiquidAI·Instruct·released 23 Oct 2026
ollama run lfm3:1.7bLFM3-1.7B is a text model for local agents and specialized applications, based on our new tri-hybrid architecture.
How much VRAM does LFM3-1.7B need?
At Q4_K_M LFM3-1.7B needs about 1.9 GB with an 8K context (1.9 GB at Q4, 4.3 GB at full precision). Pick your GPU to see what fits and how fast it runs.
Needs: ≈1.9 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.
Advanced: context 8K · KV cache 16-bit
Smallest GPU for each quant and context (KV cache 16-bit)
| Quantisation | 8K | 16K | 32K | 64K | 128K | 256K |
|---|---|---|---|---|---|---|
| Q3_K_M | ||||||
| Q4_K_M | ||||||
| Q5_K_M | ||||||
| Q6_K | ||||||
| Q8_0 | ||||||
| FP16 |
≤16 GB card24–32 GB48–96 GBmore
| Quantisation | Total |
|---|---|
| Q2_KHeavy lossSmallest, noticeably worse quality | ≈1.6 GB |
| Q3_K_MNoticeable lossSmall, some quality loss | ≈1.7 GB |
| Q4_K_MrecommendedSmall lossRecommended default for local use | ≈1.9 GB |
| Q5_K_MNear-losslessNear-lossless for most tasks | ≈2.1 GB |
| Q6_KNear-losslessPractically lossless | ≈2.3 GB |
| Q8_0LosslessLossless in practice, twice the size of Q4 | ≈2.7 GB |
| FP16 / BF16LosslessFull precision, as released | ≈4.3 GB |
Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it. Weights marked “file” use the real GGUF download size; “est.” rows are computed from the parameter count.
How to pick a quant
Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.
- LosslessF16, Q8_0
- Near-losslessQ6_K, Q5_K_M
- Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
- Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
- Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K
Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B
How much context fits?
At 8K context, Q4_K_M needs ≈1.9 GB(1.0 GB weights + 0.4 GB KV cache + 0.6 GB overhead). Pick a GPU above to see how much context fits.
Weights + overheadKV cache
Can the RTX 4060 run LFM3-1.7B? →
Speed vs quality per quant
Smaller quants run faster and fit in less memory, but lose quality.
Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.
Real-world speed of LFM3-1.7B
Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →
No medians yet. Be one of the first 3 to report a setup.
All reports
Run LFM3-1.7B locally
--quantQ4_K_Mdefault; pick your GPU to tailor itLiquidAI/LFM3-1.7B-GGUF
ollama run lfm3:1.7bThe library build picks its own default quantisation.
ollama run hf.co/LiquidAI/LFM3-1.7B-GGUF:Q4_K_M llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_MDownloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.
lms get LiquidAI/LFM3-1.7B-GGUF@Q4_K_MThen load it from the LM Studio app or with `lms load`.
pip install mlx-lm
mlx_lm.generate --model LiquidAI/LFM3-1.7B-MLX-4bit --prompt "Hello"Use `mlx_lm.chat` with the same --model for an interactive chat.
mlx_lm.server --model LiquidAI/LFM3-1.7B-MLX-4bit --port 8080API on http://localhost:8080/v1.
from transformers import pipeline
pipe = pipeline("text-generation", model="LiquidAI/LFM3-1.7B", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])Quantised versions and downloads
| Repository | Format | Quant | Size |
|---|---|---|---|
| LiquidAI/LFM3-1.7B-NVFP4 | Safetensors | NVFP4 | — |
| LiquidAI/LFM3-1.7B-FP8 | Safetensors | FP8 | — |
| bartowski/LFM3-1.7B-GGUF | GGUF | IQ2_M | 0.7 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | Q2_K | 0.7 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | IQ3_XXS | 0.8 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | IQ3_XS | 0.8 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | Q3_K_S | 0.8 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | Q3_K_M | 0.8 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | Q3_K_L | 0.9 GB |
| bartowski/LFM3-1.7B-GGUF | GGUF | IQ3_M | 0.9 GB |
About LFM3-1.7B
LFM3-1.7B is a text model for local agents and specialized applications, based on our new tri-hybrid architecture. • Agentic training: post-trained for agentic harnesses such as Hermes Agent, Pi, and OpenClaw, with a 256K-token context window. • Designed for the edge: at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop. • Made to be specialized: adapt the model to your domains, formats, and tools via quick single-GPU fine-tuning. Find more information about the LFM3 family in our release blog post. The LFM3 family includes base, text, vision, audio, omni, and decision models. LFM3-1.7B is designed for local agents, tool calling, structured outputs, and domain-specific fine-tuning. Use retrieval or search for knowledge-heavy tasks. It is not recommended for agentic coding or complex multi-step reasoning due to its small size. Choose a checkpoint variant for your inference framework and hardware. Checkpoint Framework Description BF16 vLLM , SGLang , Transformers BF16 weights for GPU inference. FP8 SGLang FP8 quantization for GPU inference. NVFP4 SGLang 4-bit floating-point quantization for GPU inference. GGUF llama.cpp , Ollama , LM Studio Local inference on CPUs and GPUs. MLX MLX 4-bit quantization for Apple Silicon. ONNX ONNX Runtime Cross-platform inference with ONNX Runtime. Sampling parameters:
Full model card on Hugging Face →Alternatives to LFM3-1.7B
All alternatives →LFM3-1.7B-Base is the pre-trained checkpoint behind LFM3-1.7B, a text model for local agents and specialized applications based on our new tri-hybrid architecture.
Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.
Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.
Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2:
We introduce Spark-X2.6-1.7B, the second model in the Spark-X2.6 series, following the release of Spark-X2.6-4B. Building on Spark-X2.5-1.7B, this release further advances our compact, general-purpose language models for on-device deployment.
LFM3-1.7B FAQ
- How much VRAM does LFM3-1.7B need?
- About 1.9 GB at Q4_K_M with an 8K context. Q8_0 needs about 2.7 GB and full precision about 4.3 GB. Longer contexts need more memory for the KV cache.
- Can LFM3-1.7B run on the RTX 4060?
- Yes. It fits at FP16 / BF16 (about 4.3 GB), at an estimated 43 tokens/s. It fits comfortably on the Apple M4 (16 GB), Radeon RX 6700 XT, RTX 3060 12GB.
- Can I use LFM3-1.7B commercially?
- Yes, with conditions. It is released under lfm1.0. Custom licence: read the model card before using it commercially.
- What is the context length of LFM3-1.7B?
- 262,144 tokens (256K). The full window needs far more memory than an 8K context.
- How do I run LFM3-1.7B with Ollama or llama.cpp?
- With Ollama: ollama run lfm3:1.7b. With llama.cpp: llama-server -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M, which starts an OpenAI-compatible server.
Rate this model
Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.
0 comments