selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

LiquidAI logo

LFM3-1.7B-Base

LiquidAI·Base model·released 26 Oct 2026

lfm1.01.7B256K ctx ≈1.9 GB
Your GPU
ollama run lfm3:1.7b
All 3 runners ↓

LFM3-1.7B-Base is the pre-trained checkpoint behind LFM3-1.7B, a text model for local agents and specialized applications based on our new tri-hybrid architecture.

How much VRAM does LFM3-1.7B-Base need?

At Q4_K_M LFM3-1.7B-Base needs about 1.9 GB with an 8K context (1.9 GB at Q4, 4.3 GB at full precision). Pick your GPU to see what fits and how fast it runs.

--vram
any
--gpus
--quant

Needs: ≈1.9 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.

Change quant, context or KV cache to compare
Advanced: context 8K · KV cache 16-bit
Context
KV cache

Smallest GPU for each quant and context (KV cache 16-bit)

Quantisation8K16K32K64K128K256K
Q3_K_M
Q4_K_M
Q5_K_M
Q6_K
Q8_0
FP16

≤16 GB card24–32 GB48–96 GBmore

QuantisationQualityWeightsKV cacheTotal
Q2_KHeavy lossSmallest, noticeably worse qualityHeavy loss0.7 GB file0.4 GB≈1.6 GB
Q3_K_MNoticeable lossSmall, some quality lossNoticeable loss0.8 GB file0.4 GB≈1.8 GB
Q4_K_MrecommendedSmall lossRecommended default for local useSmall loss1.0 GB file0.4 GB≈1.9 GB
Q5_K_MNear-losslessNear-lossless for most tasksNear-lossless1.2 GB file0.4 GB≈2.1 GB
Q6_KNear-losslessPractically losslessNear-lossless1.3 GB file0.4 GB≈2.3 GB
Q8_0LosslessLossless in practice, twice the size of Q4Lossless1.7 GB file0.4 GB≈2.7 GB
FP16 / BF16LosslessFull precision, as releasedLossless3.2 GB file0.4 GB≈4.3 GB

Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it. Weights marked “file” use the real GGUF download size; “est.” rows are computed from the parameter count.

How to pick a quant

Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.

  • LosslessF16, Q8_0
  • Near-losslessQ6_K, Q5_K_M
  • Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
  • Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
  • Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K

Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B

How much context fits?

Model quant
KV cache

At 8K context, Q4_K_M needs ≈1.9 GB(1.0 GB weights + 0.4 GB KV cache + 0.6 GB overhead). Pick a GPU above to see how much context fits.

0510151K2K4K8K16K32K64K128K256KGB

Weights + overheadKV cache

Can the RTX 4060 run LFM3-1.7B-Base? →

Speed vs quality per quant

Smaller quants run faster and fit in less memory, but lose quality.

Q2_KQ3_K_MQ4_K_MQ5_K_MQ6_KQ8_0F16

Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.

Real-world speed of LFM3-1.7B-Base

Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →

No medians yet. Be one of the first 3 to report a setup.

All reports

Run LFM3-1.7B-Base locally

--quantQ4_K_Mdefault; pick your GPU to tailor itmradermacher/LFM3-1.7B-Base-GGUF

From the Ollama librarydocs
ollama run lfm3:1.7b

The library build picks its own default quantisation.

Q4_K_M straight from Hugging Facedocs
ollama run hf.co/mradermacher/LFM3-1.7B-Base-GGUF:Q4_K_M

Install Ollama

OpenAI-compatible server with a web UIdocs
llama-server -hf mradermacher/LFM3-1.7B-Base-GGUF:Q4_K_M

Downloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.

Install llama.cpp

Chat with the pipeline APIdocs
python
from transformers import pipeline

pipe = pipeline("text-generation", model="LiquidAI/LFM3-1.7B-Base", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])

Install Transformers

Quantised versions and downloads

RepositoryFormatQuantSizeDownloads
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ2_K0.8 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ3_K_S0.9 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ3_K_M0.9 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ3_K_L1.0 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFIQ4_XS1.0 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ4_K_S1.1 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ4_K_M1.1 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ5_K_S1.2 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ5_K_M1.3 GB313
mradermacher/LFM3-1.7B-Base-GGUFGGUFQ6_K1.4 GB313

About LFM3-1.7B-Base

LFM3-1.7B-Base is the pre-trained checkpoint behind LFM3-1.7B, a text model for local agents and specialized applications based on our new tri-hybrid architecture. • Pre-trained on 22T tokens: the last 3.5T tokens are curated mid-training data with agentic traces and tool calls, and extend the context window to 256K tokens. • Designed for the edge: at 4-bit precision, it takes about 1 GB of memory, so it fits on any phone or laptop. • Made to be specialized: a starting point for continued pre-training and your own post-training. Find more information about the LFM3 family in our release blog post. The LFM3 family includes base, text, vision, audio, omni, and decision models. LFM3-1.7B-Base is the checkpoint used to create LFM3-1.7B. It is only recommended for tasks that require heavy fine-tuning, like language-specific or domain-specific assistants, training on proprietary data, or experimenting with novel post-training approaches. For chat, tool calling, and agents, use LFM3-1.7B. LFM3-1.7B-Base uses the same architecture and modeling code as LFM3-1.7B, with BF16 weights. Transformers quick start: Mid-training included chat data with and without thinking traces, formatted with the same chat template as LFM3-1.7B. With apply_chat_template, reasoning is enabled by default, so responses start with a thinking trace. Set enable_thinking=False to prefill an empty thinking trace and have the model answer directly.

Full model card on Hugging Face →

Alternatives to LFM3-1.7B-Base

All alternatives →
openbmb logo

MiniCPM5-2B

openbmb·Chat

≈2.3 GB

We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

2.5B128K ctx Apache 2.0
1MSept 2026gguf
Qwen logo

Qwen3-1.7B

Qwen·Chat

≈2.6 GB

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.

2B40K ctx Apache 2.0
3MApr 2025ollama
Qwen logo

Qwen3-1.7B-Base

Qwen·Chat

≈2.4 GB

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.

1.7B32K ctx Apache 2.0
618.8KApr 2025gguf
Qwen logo≈1.6 GB

Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2:

1.5B32K ctx Apache 2.0
5.1MSept 2024ollama
XHToken logo

Spark-X2.6-1.7B

XHToken·Chat

≈1.9 GB

We introduce Spark-X2.6-1.7B, the second model in the Spark-X2.6 series, following the release of Spark-X2.6-4B. Building on Spark-X2.5-1.7B, this release further advances our compact, general-purpose language models for on-device deployment.

1.7B1M ctx Apache 2.0
5.6KOct 2026gguf
tencent logo≈1.9 GB

This checkpoint is a five-layer draft model for Qwen3.8-27B, with block size 16 and an 8,192-token sliding window.

2B256K ctx Apache 2.0
5.6KNov 2026gguf

LFM3-1.7B-Base FAQ

How much VRAM does LFM3-1.7B-Base need?
About 1.9 GB at Q4_K_M with an 8K context. Q8_0 needs about 2.7 GB and full precision about 4.3 GB. Longer contexts need more memory for the KV cache.
Can LFM3-1.7B-Base run on the RTX 4060?
Yes. It fits at FP16 / BF16 (about 4.3 GB), at an estimated 43 tokens/s. It fits comfortably on the Apple M4 (16 GB), Radeon RX 6700 XT, RTX 3060 12GB.
Can I use LFM3-1.7B-Base commercially?
Yes, with conditions. It is released under lfm1.0. Custom licence: read the model card before using it commercially.
What is the context length of LFM3-1.7B-Base?
262,144 tokens (256K). The full window needs far more memory than an 8K context.
How do I run LFM3-1.7B-Base with Ollama or llama.cpp?
With Ollama: ollama run lfm3:1.7b. With llama.cpp: llama-server -hf mradermacher/LFM3-1.7B-Base-GGUF:Q4_K_M, which starts an OpenAI-compatible server.

Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.