selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

Alternatives to Qwen3.8-4B-Distill

120 open-weight models that do the same job as Qwen3.8-4B-Distill (4.7B chat), ranked by how close they are in task and size, then by popularity and how recent they are. Licence and VRAM are shown for each.

the original
empero-ai logo

Qwen3.8-4B-Distill

empero-ai·Chat

≈4.3 GB

Qwen3.8-4B is a full-parameter distillation of Qwen3.8 2.4T A95B into the Qwen3.5-4B architecture. The student was trained on ~45,000 curated teacher traces from our internal Qwen3.8 distillation datasets — dense chain-of-thought spanning mathematics, general reasoning, and instruction following…

4.7B256K ctx Apache 2.0
130.3KAug 2026gguf
--vram
any
--gpus
--quant
  1. openbmb logo

    AgentCPM-Explore

    openbmb·Chat

    ≈4.0 GB

    We elaborate on the entire construction pipeline of AgentCPM-Explore on GitHub.

    4B256K ctx Apache 2.0
    2.3KJan 2026gguf
    • Same category
    • Similar size (4B vs 4.7B)
    • Commercial use OK
    vs
  2. ibm-granite logo

    granite-4.0-micro

    ibm-granite·Chat

    ≈3.1 GB

    📣 Update [10-07-2025]: Added a default system prompt to the chat template to guide the model towards more professional, accurate, and safe responses.

    3.4B128K ctx Apache 2.0
    23.8KSept 2025gguf
    • Same category
    • Similar size (3.4B vs 4.7B)
    • Commercial use OK
    vs
  3. ibm-granite logo

    granite-4.1-8b

    ibm-granite·Chat

    ≈7.0 GB

    Model Summary: Granite-4.1-8B is a 8B parameter long-context instruct model finetuned from Granite-4.1-8B-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets.

    8.8B128K ctx Apache 2.0
    83.6KApr 2026ollama
    • Same category
    • Similar size (8.8B vs 4.7B)
    • Commercial use OK
    vs
  4. meta-llama logo

    Llama-3.1-8B-Instruct

    meta-llama·Chat

    ≈6.3 GB

    The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models in 8B, 70B and 405B sizes (text in/text out).

    8B128K ctx Llama 3.1 Community License
    4.2MJul 2024ollama
    • Same category
    • Similar size (8B vs 4.7B)
    • Widely used
    vs
  5. Qwen logo

    Qwen3-8B-Base

    Qwen·Chat

    ≈6.5 GB

    Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.

    8.2B32K ctx Apache 2.0
    600.1KApr 2025gguf
    • Same category
    • Similar size (8.2B vs 4.7B)
    • Commercial use OK
    vs
  6. swiss-ai logo≈6.3 GB

    Apertus is a 70B and 8B parameter language model designed to push the boundaries of fully-open multilingual and transparent models.

    8.1B64K ctx Apache 2.0
    218.5KAug 2025gguf
    • Same category
    • Similar size (8.1B vs 4.7B)
    • Commercial use OK
    vs
  7. IFM logo

    K2-Horizon-7B

    IFM·Chat

    ≈7.0 GB

    K2-Horizon-7B is the medium dense member of the K2-Horizon family: a 7B-core decoder-only model with a 512K context window.

    9B512K ctx Apache 2.0
    31KSept 2026gguf
    • Same category
    • Similar size (9B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  8. nvidia logo≈3.4 GB

    Nemotron-Mini-4B-Instruct is a model for generating responses for roleplaying, retrieval augmented generation, and function calling. It is a small language model (SLM) optimized through distillation, pruning and quantization for speed and on-device deployment.

    4B4K ctx NVIDIA Open Model License
    78.5KSept 2024ollama
    • Same category
    • Similar size (4B vs 4.7B)
    • Commercial use OK
    vs
  9. microsoft logo

    Sentiment-Analysis-7B

    microsoft·Chat

    ≈5.8 GB

    Sentiment-Analysis-7B is a 7B-parameter autoregressive language model fine-tuned for sentiment classification of user messages directed at a service, product, or agent (the "Target").

    7.2B32K ctx MIT
    3KOct 2026gguf
    • Same category
    • Similar size (7.2B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  10. arcee-ai logo

    AFM-4.5B-Base

    arcee-ai·Chat

    ≈3.8 GB

    AFM-4.5B-Base is a 4.5 billion parameter instruction-tuned model developed by Arcee.ai, designed for enterprise-grade performance across diverse deployment environments from cloud to edge.

    4.6B64K ctx Apache 2.0
    1.6KJul 2025gguf
    • Same category
    • Similar size (4.6B vs 4.7B)
    • Commercial use OK
    vs
  11. HuggingFaceTB logo

    SmolLM3-3B-Base

    HuggingFaceTB·Chat

    ≈2.9 GB

    SmolLM3 is a 3B parameter language model designed to push the boundaries of small models. It supports 6 languages, advanced reasoning and long context. SmolLM3 is a fully open model that offers strong performance at the 3B–4B scale.

    3.1B64K ctx Apache 2.0
    87.3KJun 2025gguf
    • Same category
    • Similar size (3.1B vs 4.7B)
    • Commercial use OK
    vs
  12. microsoft logo

    Phi-3-mini-4k-instruct

    microsoft·Chat

    ≈4.3 GB

    The Phi-3-Mini-4K-Instruct is a 3.8B parameters, lightweight, state-of-the-art open model trained with the Phi-3 datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties.

    3.8B4K ctx MIT
    304.9KApr 2024ollama
    • Same category
    • Similar size (3.8B vs 4.7B)
    • Commercial use OK
    vs
  13. CohereLabs logo

    tiny-aya-earth

    CohereLabs·Chat

    ≈3.1 GB

    Cohere Labs Tiny Aya is an open weights research release of a pretrained 3.35 billion parameter model optimized for efficient, strong, and balanced multilingual representation across 70+ languages, including many lower-resourced ones.

    3.3BCC BY-NC 4.0
    5.9KFeb 2026gguf
    • Same category
    • Similar size (3.3B vs 4.7B)
    vs
  14. A compact 9B reasoning model distilled from DeepSeek-V4-Pro in Max Effect mode

    9.4B256K ctx Apache 2.0
    24.5KOct 2026gguf
    • Same category
    • Larger (9.4B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  15. nvidia logo≈6.6 GB

    Nemotron-Labs-Diffusion is a tri-mode language model that supports both AR decoding and diffusion-based parallel decoding by simply switching the attention pattern of the same model during inference.

    8.5B256K ctx nvidia-nemotron-open-model-license
    49.7KMar 2026gguf
    • Same category
    • Similar size (8.5B vs 4.7B)
    • Commercial use OK
    vs
  16. ServiceNow-AI logo

    Apriel-5B-Instruct

    ServiceNow-AI·Chat

    ≈4.2 GB

    Apriel is a family of models built for versatility, offering high throughput and efficiency across a wide range of tasks.

    4.8B16K ctx MIT
    3.9KApr 2025
    • Same category
    • Similar size (4.8B vs 4.7B)
    • Commercial use OK
    vs
  17. microsoft logo≈5.8 GB

    The Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter, lightweight, state-of-the-art open model trained using the Phi-3 datasets. This dataset includes both synthetic data and filtered publicly available website data, with an emphasis on high-quality and reasoning-dense properties.

    3.8B128K ctx MIT
    240.2KApr 2024gguf
    • Same category
    • Similar size (3.8B vs 4.7B)
    • Commercial use OK
    vs
  18. ibm-granite logo

    granite-4.0-h-tiny

    ibm-granite·Chat

    ≈5.2 GB

    📣 Update [10-07-2025]: Added a default system prompt to the chat template to guide the model towards more professional, accurate, and safe responses.

    6.9BA1.5B128K ctx Apache 2.0
    27.4KSept 2025gguf
    • Same category
    • Similar size (6.9B vs 4.7B)
    • Commercial use OK
    vs
  19. meta-llama logo

    Llama-3.2-3B

    meta-llama·Chat

    ≈3.3 GB

    The Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out).

    3.2B128K ctx Llama 3.2 Community License
    317KSept 2024gguf
    • Same category
    • Similar size (3.2B vs 4.7B)
    vs
  20. openbmb logo

    MiniCPM5-2B-Base

    openbmb·Chat

    ≈2.3 GB

    We are releasing MiniCPM5-2B, the second model in the MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B Transformer that scales up the same training recipe, built for on-device, local deployment, and resource-constrained scenarios, reaching 2B-class open-source SOTA.

    2.5B512K ctx Apache 2.0
    12.7KAug 2026gguf
    • Same category
    • Similar size (2.5B vs 4.7B)
    • Commercial use OK
    • Newer release
    vs
  21. Qwen logo≈6.5 GB

    Qwen3Guard is a series of safety moderation models built upon Qwen3 and trained on a dataset of 1.19 million prompts and responses labeled for safety.

    8.2B32K ctx Apache 2.0
    110.5KSept 2025gguf
    • Same category
    • Similar size (8.2B vs 4.7B)
    • Commercial use OK
    vs
  22. LiquidAI logo

    LFM2.5-8B-A1B

    LiquidAI·Chat

    ≈5.9 GB

    LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.

    8.5BA1B125K ctx lfm1.0
    19.7KMay 2026ollama
    • Same category
    • Similar size (8.5B vs 4.7B)
    vs
  23. speakleash logo≈4.1 GB

    Bielik-4.5B-v3-Instruct is a generative text model featuring 4.6 billion parameters. It is an instruct fine-tuned version of the Bielik-4.5B-v3.

    4.8BApache 2.0
    2.4KApr 2025gguf
    • Same category
    • Similar size (4.8B vs 4.7B)
    • Commercial use OK
    vs
  24. ibm-granite logo≈3.0 GB

    Model Summary: Granite-3.1-3B-A800M-Instruct is a 3B parameter long-context instruct model finetuned from Granite-3.1-3B-A800M-Base using a combination of open source instruction datasets with permissive license and internally collected synthetic datasets tailored for solving long context problems.

    3.3BA800M128K ctx Apache 2.0
    114.9KDec 2024gguf
    • Same category
    • Similar size (3.3B vs 4.7B)
    • Commercial use OK
    vs