selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

FINAL-Bench logo

Darwin-27B-RSI

FINAL-Bench·Instruct·released 27 Sept 2026

Apache 2.026.9B256K ctx ≈18 GB
Your GPU
ollama run hf.co/FINAL-Bench/Darwin-27B-RSI-GGUF:Q4_K_M
All 6 runners ↓

Qwen3.5-27B family · 27B dense · Thinking mode · BF16 · Apache 2.0 No human-written answers. The model generated its own learning signal — and got measurably better.

Notify me when

How much VRAM does Darwin-27B-RSI need?

At Q4_K_M Darwin-27B-RSI needs about 18 GB with an 8K context (18 GB at Q4, 55 GB at full precision). Pick your GPU to see what fits and how fast it runs.

--vram
any
--gpus
--quant

Needs: ≈19 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.

Change quant, context or KV cache to compare
Advanced: context 8K · KV cache 16-bit
Context
KV cache

Smallest GPU for each quant and context (KV cache 16-bit)

Quantisation8K16K32K64K128K256K
Q3_K_M
Q4_K_M
Q5_K_M
Q6_K
Q8_0
FP16

≤16 GB card24–32 GB48–96 GBmore

QuantisationQualityWeightsKV cacheTotal
Q2_KHeavy lossSmallest, noticeably worse qualityHeavy loss9.9 GB file2.0 GB≈13 GB
Q3_K_MNoticeable lossSmall, some quality lossNoticeable loss12 GB file2.0 GB≈15 GB
Q4_K_MrecommendedSmall lossRecommended default for local useSmall loss16 GB file2.0 GB≈19 GB
Q5_K_MNear-losslessNear-lossless for most tasksNear-lossless19 GB file2.0 GB≈23 GB
Q6_KNear-losslessPractically losslessNear-lossless22 GB file2.0 GB≈26 GB
Q8_0LosslessLossless in practice, twice the size of Q4Lossless27 GB file2.0 GB≈31 GB
FP16 / BF16LosslessFull precision, as releasedLossless50 GB file2.0 GB≈55 GB

Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it. Weights marked “file” use the real GGUF download size; “est.” rows are computed from the parameter count.

How to pick a quant

Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.

  • LosslessF16, Q8_0
  • Near-losslessQ6_K, Q5_K_M
  • Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
  • Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
  • Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K

Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B

How much context fits?

Model quant
KV cache

At 8K context, Q4_K_M needs ≈19 GB(16 GB weights + 2.0 GB KV cache + 1.3 GB overhead). Pick a GPU above to see how much context fits.

0204060801001K2K4K8K16K32K64K128K256KGB

Weights + overheadKV cache

Can the Tesla P40 run Darwin-27B-RSI? →

Speed vs quality per quant

Smaller quants run faster and fit in less memory, but lose quality.

Q2_KQ3_K_MQ4_K_MQ5_K_MQ6_KQ8_0F16

Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.

Real-world speed of Darwin-27B-RSI

Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →

No medians yet. Be one of the first 3 to report a setup.

All reports

Run Darwin-27B-RSI locally

--quantQ4_K_Mdefault; pick your GPU to tailor itFINAL-Bench/Darwin-27B-RSI-GGUF

Q4_K_M straight from Hugging Facedocs
ollama run hf.co/FINAL-Bench/Darwin-27B-RSI-GGUF:Q4_K_M

Install Ollama

OpenAI-compatible server with a web UIdocs
llama-server -hf FINAL-Bench/Darwin-27B-RSI-GGUF:Q4_K_M

Downloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.

Install llama.cpp

Download with the lms CLIdocs
lms get FINAL-Bench/Darwin-27B-RSI-GGUF@Q4_K_M

Then load it from the LM Studio app or with `lms load`.

Install LM Studio

Serve an OpenAI-compatible APIdocs
vllm serve FINAL-Bench/Darwin-27B-RSI
Docker Compose (NVIDIA)docs
yaml
services:
  vllm:
    image: vllm/vllm-openai:latest
    command: FINAL-Bench/Darwin-27B-RSI
    ports:
      - "8000:8000"
    ipc: host
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

API on port 8000.

Install vLLM

Serve an OpenAI-compatible APIdocs
sglang serve FINAL-Bench/Darwin-27B-RSI --host 0.0.0.0 --port 30000
Docker Compose (NVIDIA)docs
yaml
services:
  sglang:
    image: lmsysorg/sglang:latest
    command: sglang serve FINAL-Bench/Darwin-27B-RSI --host 0.0.0.0 --port 30000
    ports:
      - "30000:30000"
    ipc: host
    volumes:
      - ~/.cache/huggingface:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

API on port 30000.

Install SGLang

Chat with the pipeline APIdocs
python
from transformers import pipeline

pipe = pipeline("text-generation", model="FINAL-Bench/Darwin-27B-RSI", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])

Install Transformers

  • Jancommunity · desktop app
  • KoboldCpp logoKoboldCppcommunity · desktop app
  • LocalAI logoLocalAIcommunity · inference server

Quantised versions and downloads

About Darwin-27B-RSI

Qwen3.5-27B family · 27B dense · Thinking mode · BF16 · Apache 2.0 No human-written answers. The model generated its own learning signal — and got measurably better. --- Darwin-27B-RSI is Darwin-27B-Opus after Recursive Self-Improvement (RSI): the model was improved using only signal it produced itself. During self-improvement, the model itself (its weights) improves by learning only from its own solutions. No human-written solutions or reasoning traces are used; correctness is checked automatically (agreement across its own samples and code execution). Under an identical evaluation protocol, Darwin-27B-RSI improves over its parent on graduate-level science reasoning — +5.24 points on GPQA Diamond (single sample) and +3.79 points with majority voting — with every gain statistically significant in paired tests. As the reasoning engine of Darwin-27B-JEV on the Decision Index, it lifts the hardest reasoning decisions: GPQA Diamond skill 0.31 → 0.71, GSM8K 0.61 → 0.97, MMLU-Pro 0.60 → 0.82. --- Darwin-27B-RSI is Model-level RSI: the model itself (its weights) improves by learning only from its own solutions. No human-written solutions or reasoning traces are used; correctness is checked automatically. You download a new model file, and it is smarter on its own.

Full model card on Hugging Face →

Alternatives to Darwin-27B-RSI

All alternatives →

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

31.6BA3B256K ctx openmdw-1.1
963.1KAug 2026gguf
farbodtavakkoli logo

OTel-2.0-LLM-31B-IT

farbodtavakkoli·Chat

≈27 GB

Versioning notice: This card describes the current OTel 2.0 OSFT checkpoint. The model weights may be updated over time. For reproducible evaluation or production deployment, pin a specific model revision, checkpoint hash, or release tag rather than relying on the floating latest version.

31.3B256K ctx Apache 2.0
866.9KJul 2026gguf
ornith-ai logo

Ornith-1.0-35B

ornith-ai·Chat

≈22 GB

Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.

35BA3.8B256K ctx MIT
1.5MJun 2026gguf
zai-org logo

GLM-4.7-Flash

zai-org·Chat

≈26 GB

👋 Join our Discord community. 📖 Check out the GLM-4.7 technical blog , technical report(GLM-4.5) . 📍 Use GLM-4.7-Flash API services on Z.ai API Platform. 👉 One click to GLM-4.7 .

31.2BA4.6B198K ctx MIT
1.1MJan 2026ollama
poolside logo

Laguna-XS-2.1

poolside·Chat

≈22 GB

Laguna XS 2.1 is a 33B total parameter Mixture-of-Experts model with 3B activated parameters per token designed for agentic coding and long-horizon work on a local machine.

33.4BA2.2B256K ctx Custom licence
587.1KJun 2026ollama

Nemotron-3-Nano-30B-A3B-BF16 is a large language model (LLM) trained from scratch by NVIDIA, and designed as a unified model for both reasoning and non-reasoning tasks. It responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response.

31.6BA3B256K ctx nvidia-nemotron-open-model-license
1.4MDec 2025gguf

Darwin-27B-RSI FAQ

How much VRAM does Darwin-27B-RSI need?
About 18 GB at Q4_K_M with an 8K context. Q8_0 needs about 30 GB and full precision about 55 GB. Longer contexts need more memory for the KV cache.
Can Darwin-27B-RSI run on the Tesla P40?
Yes. It fits at Q5_K_M (about 23 GB), at an estimated 10 tokens/s. It fits comfortably on the Radeon RX 7900 XTX, Instinct MI50 32GB, RTX 5090.
Can I use Darwin-27B-RSI commercially?
Yes. It is released under Apache 2.0. Permissive: use, modify and sell freely; keep the notice and licence text.
What is the context length of Darwin-27B-RSI?
262,144 tokens (256K). The full window needs far more memory than an 8K context.
How do I run Darwin-27B-RSI with Ollama or llama.cpp?
With Ollama: ollama run hf.co/FINAL-Bench/Darwin-27B-RSI-GGUF:Q4_K_M. With llama.cpp: llama-server -hf FINAL-Bench/Darwin-27B-RSI-GGUF:Q4_K_M, which starts an OpenAI-compatible server.

Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.