selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

Alternatives to Qwen3.8-4B-Distill

120 open-weight models that do the same job as Qwen3.8-4B-Distill (4.7B chat), ranked by how close they are in task and size, then by popularity and how recent they are. Licence and VRAM are shown for each.

the original
empero-ai logo

Qwen3.8-4B-Distill

empero-ai·Chat

≈4.3 GB

Qwen3.8-4B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture. The student was trained on ~45,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, general reasoning, and instruction following…

4.7B256K ctx Apache 2.0
112.4KAug 2026gguf
--vram
any
--gpus
--quant
  1. tomg-group-umd logo

    huginn-0125

    tomg-group-umd·Chat

    ≈4.1 GB

    This is Huginn, version 01/25, a latent recurrent-depth model with 3.5B parameters, trained for 800B tokens on AMD MI250X machines. This is a proof-of-concept model, but surprisingly capable in reasoning and code given its training budget and size.

    3.9BApache 2.0
    20.7KJan 2025
    • Same category
    • Similar size (3.9B vs 4.7B)
    • Commercial use OK
    vs
  2. ibm-granite logo≈3.0 GB

    Model Summary: Granite-3.1-3B-A800M-Instruct is a 3B parameter long-context instruct model finetuned from Granite-3.1-3B-A800M-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems.

    3.3BA800M128K ctx Apache 2.0
    114.9KDec 2024gguf
    • Same category
    • Similar size (3.3B vs 4.7B)
    • Commercial use OK
    vs
  3. Qwen logo

    Qwen3-1.7B

    Qwen·Chat

    ≈2.6 GB

    Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.

    2B40K ctx Apache 2.0
    3MApr 2025ollama
    • Same category
    • Smaller (2B vs 4.7B)
    • Commercial use OK
    • Widely used
    vs
  4. ibm-granite logo

    granite-swash-3b-a600m

    ibm-granite·Chat

    ≈2.5 GB

    Model Summary Granite-SWASH-3B-a600M is a 3B parameter decoder-only sparse language model with per-layer Sliding Window Attention (SWA) and learnable attention sinks (LSE-based).

    3BA600M8K ctx Apache 2.0
    3KJul 2026gguf
    • Same category
    • Similar size (3B vs 4.7B)
    • Commercial use OK
    vs
  5. LiquidAI logo

    LFM2.5-8B-A1B

    LiquidAI·Chat

    ≈5.9 GB

    LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.

    8.5BA1B125K ctx lfm1.0
    17KMay 2026ollama
    • Same category
    • Similar size (8.5B vs 4.7B)
    vs
  6. WeiboAI logo

    VibeThinker-3B

    WeiboAI·Chat

    ≈2.6 GB

    2.For harder math reasoning, try AMOBench , a problem set harder than the International Mathematical Olympiad (IMO), with included standard answers. Use it to evaluate VibeThinker against other SOTA models. Note: due to extreme difficulty, set max tokens to 60K–100K.

    3.1B128K ctx MIT
    2.6KJun 2026gguf
    • Same category
    • Similar size (3.1B vs 4.7B)
    • Commercial use OK
    vs
  7. allenai logo

    Olmo-Hybrid-7B

    allenai·Chat

    ≈8.7 GB

    We expand on our Olmo model series by introducing Olmo Hybrid, a new 7B hybrid RNN model in the Olmo family. Olmo Hybrid dramatically outperforms Olmo 3 in final performance, consistently showing roughly 2x data efficiency on core evals over the course of our pretraining run.

    7.4B64K ctx Apache 2.0
    10.9KJan 2026
    • Same category
    • Similar size (7.4B vs 4.7B)
    • Commercial use OK
    vs
  8. CohereLabs logo

    tiny-aya-base

    CohereLabs·Chat

    ≈3.1 GB

    Cohere Labs Tiny Aya is an open weights research release of a pretrained 3.35 billion parameter model optimized for efficient, strong, and balanced multilingual representation across 70+ languages, including many lower-resourced ones.

    3.3BCC BY-NC 4.0
    2.9KFeb 2026gguf
    • Same category
    • Similar size (3.3B vs 4.7B)
    vs
  9. arcee-ai logo

    Trinity-Nano-Preview

    arcee-ai·Chat

    ≈4.6 GB

    Trinity Nano Preview is a preview of Arcee AI's 6B MoE model with 1B active parameters. It is the small-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.

    6.1BA1B128K ctx openmdw-1.1
    2.6KDec 2025gguf
    • Same category
    • Similar size (6.1B vs 4.7B)
    • Commercial use OK
    vs
  10. ibm-granite logo

    granite-switch-4.1-8b

    ibm-granite·Chat

    ≈7.0 GB

    📝 Read the launch blog: Granite Switch 4.1: Chat, Decisions, and Speech from One Endpoint

    8.8B128K ctx Apache 2.0
    6.2KOct 2026gguf
    • Same category
    • Similar size (8.8B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  11. LGAI-EXAONE logo

    EXAONE-3.5-7.8B-Instruct

    LGAI-EXAONE·Chat

    ≈6.1 GB

    We introduce EXAONE 3.5, a collection of instruction-tuned bilingual (English and Korean) generative models ranging from 2.4B to 32B parameters, developed and released by LG AI Research.

    7.8B32K ctx exaone
    383.6KDec 2024ollama
    • Same category
    • Similar size (7.8B vs 4.7B)
    vs
  12. openbmb logo≈2.5 GB

    This is the RL initialization checkpoint used in the blog JustRL II: Scaling Small LLMs to 128K Reasoning with a Critic (中文版).

    2.8B64K ctx Unknown licence
    2.2KSept 2026
    • Same category
    • Similar size (2.8B vs 4.7B)
    • Newer release
    vs
  13. Qwen logo

    Qwen2.5-7B

    Qwen·Chat

    ≈5.5 GB

    Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2:

    7.6B128K ctx Apache 2.0
    517KSept 2024gguf
    • Same category
    • Similar size (7.6B vs 4.7B)
    • Commercial use OK
    vs
  14. openbmb logo

    BitCPM-CANN-3B

    openbmb·Chat

    ≈2.5 GB

    BitCPM-CANN is the first end-to-end 1.58-bit (ternary) large language model training system natively built on Huawei Ascend NPU.

    3B32K ctx Apache 2.0
    3KMay 2026gguf
    • Same category
    • Similar size (3B vs 4.7B)
    • Commercial use OK
    vs
  15. openbmb logo

    MiniCPM5-2B-SFT

    openbmb·Chat

    ≈2.3 GB

    We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

    2.5B128K ctx Apache 2.0
    6.6KAug 2026
    • Same category
    • Similar size (2.5B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  16. ai21labs logo

    AI21-Jamba2-3B

    ai21labs·Chat

    ≈2.4 GB

    Jamba2 3B is an ultra-compact open source model designed to bring enterprise-grade reliability to on-device deployments. At just 3B parameters, it runs efficiently on consumer devices—iPhones, Androids, Macs, and PCs—while maintaining the grounding and instruction-following capabilities required…

    3B256K ctx Apache 2.0
    6.6KJan 2026gguf
    • Same category
    • Similar size (3B vs 4.7B)
    • Commercial use OK
    vs
  17. ibm-granite logo

    granite-4.1-8b-base

    ibm-granite·Chat

    ≈6.7 GB

    Model Summary: Granite‑4.1‑8B‑Base is a decoder‑only language model with long‑context capabilities, designed to support a broad range of text‑to‑text generation tasks.

    8.4B128K ctx Apache 2.0
    14.4KApr 2026gguf
    • Same category
    • Similar size (8.4B vs 4.7B)
    • Commercial use OK
    vs
  18. Qwen logo

    Qwen2.5-3B

    Qwen·Chat

    ≈2.6 GB

    Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2:

    3.1B32K ctx Qwen Research License
    218.8KSept 2024gguf
    • Same category
    • Similar size (3.1B vs 4.7B)
    vs
  19. arcee-ai logo

    Trinity-Nano-Base

    arcee-ai·Chat

    ≈4.6 GB

    Trinity Nano is an Arcee AI 6B MoE model with 1B active parameters. It is the small-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.

    6.1BA1B128K ctx openmdw-1.1
    1.9KDec 2025gguf
    • Same category
    • Similar size (6.1B vs 4.7B)
    • Commercial use OK
    vs
  20. nvidia logo≈7.5 GB

    NVIDIA-Nemotron-Nano-9B-v2 is a large language model (LLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. It responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response.

    8.9B128K ctx NVIDIA Open Model License
    130.2KAug 2025gguf
    • Same category
    • Similar size (8.9B vs 4.7B)
    • Commercial use OK
    vs
  21. tencent logo

    Hunyuan-4B-Instruct

    tencent·Chat

    ≈4.1 GB

    Hunyuan is Tencent's open-source efficient large language model series, designed for versatile deployment across diverse computational environments.

    4.2B256K ctx Unknown licence
    987Jul 2025gguf
    • Same category
    • Similar size (4.2B vs 4.7B)
    vs
  22. openbmb logo≈2.3 GB

    We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

    2.5B128K ctx Apache 2.0
    4.8KSept 2026
    • Same category
    • Similar size (2.5B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  23. Salesforce logo

    Llama-xLAM-2-8b-fc-r

    Salesforce·Chat

    ≈6.3 GB

    Large Action Models (LAMs) are advanced language models designed to enhance decision-making by translating user intentions into executable actions.

    8B128K ctx CC BY-NC 4.0
    137.8KMar 2025gguf
    • Same category
    • Similar size (8B vs 4.7B)
    vs
  24. LiquidAI logo

    LFM3-1.7B

    LiquidAI·Chat

    ≈1.9 GB

    LFM3-1.7B is a text model for local agents and specialized applications, based on our new tri-hybrid architecture.

    1.7B256K ctx lfm1.0
    103.7KOct 2026ollama
    • Same category
    • Smaller (1.7B vs 4.7B)
    • Newer release
    vs