selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

RWKV logo

RWKV7-G1k-13.3B-20260930

RWKV·Instruct·released 30 Sept 2026

Apache 2.013.3B≈16 GB
Your GPU
hf download RWKV/RWKV7-G1k-13.3B-20260930
Setup ↓

This is an official BlinkDL release of RWKV-7 Goose in Hugging Face Transformers format. RWKV-7 is an attention-free recurrent architecture with a constant-size recurrent state and constant inference work per generated token. Training remains parallelizable.

Notify me when

How much VRAM does RWKV7-G1k-13.3B-20260930 need?

At Q4_K_M RWKV7-G1k-13.3B-20260930 needs about 16 GB with an 8K context (16 GB at Q4, 34 GB at full precision). Pick your GPU to see what fits and how fast it runs.

--vram
any
--gpus
--quant

Needs: ≈16 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.

Change quant, context or KV cache to compare
Advanced: context 8K · KV cache 16-bit
Context
KV cache

Smallest GPU for each quant and context (KV cache 16-bit)

Quantisation4K8K16K32K64K128K
Q3_K_M
Q4_K_M
Q5_K_M
Q6_K
Q8_0
FP16

≤16 GB card24–32 GB48–96 GBmore

QuantisationQualityWeightsKV cacheTotal
Q2_KHeavy lossSmallest, noticeably worse qualityHeavy loss4.1 GB 7.6 GB≈12 GB
Q3_K_MNoticeable lossSmall, some quality lossNoticeable loss6.0 GB 7.6 GB≈14 GB
Q4_K_MrecommendedSmall lossRecommended default for local useSmall loss7.5 GB 7.6 GB≈16 GB
Q5_K_MNear-losslessNear-lossless for most tasksNear-lossless8.8 GB 7.6 GB≈17 GB
Q6_KNear-losslessPractically losslessNear-lossless10 GB 7.6 GB≈19 GB
Q8_0LosslessLossless in practice, twice the size of Q4Lossless13 GB 7.6 GB≈22 GB
FP16 / BF16LosslessFull precision, as releasedLossless25 GB 7.6 GB≈34 GB

Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it.

How to pick a quant

Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.

  • LosslessF16, Q8_0
  • Near-losslessQ6_K, Q5_K_M
  • Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
  • Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
  • Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K

Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B

How much context fits?

Model quant
KV cache

At 8K context, Q4_K_M needs ≈16 GB(7.5 GB weights + 7.6 GB KV cache + 0.9 GB overhead). Pick a GPU above to see how much context fits.

0501001501K2K4K8K16K32K64K128KGB

Weights + overheadKV cache

Can the Tesla P40 run RWKV7-G1k-13.3B-20260930? →

Speed vs quality per quant

Smaller quants run faster and fit in less memory, but lose quality.

Q2_KQ3_K_MQ4_K_MQ5_K_MQ6_KQ8_0F16

Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.

Real-world speed of RWKV7-G1k-13.3B-20260930

Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →

No medians yet. Be one of the first 3 to report a setup.

All reports

Run RWKV7-G1k-13.3B-20260930 locally

These commands load the original weights from Hugging Face.

Chat with the pipeline APIdocs
python
from transformers import pipeline

pipe = pipeline("text-generation", model="RWKV/RWKV7-G1k-13.3B-20260930", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])

Install Transformers

About RWKV7-G1k-13.3B-20260930

This is an official BlinkDL release of RWKV-7 Goose in Hugging Face Transformers format. RWKV-7 is an attention-free recurrent architecture with a constant-size recurrent state and constant inference work per generated token. Training remains parallelizable. This checkpoint is a base model pretrained with web, code, synthetic, instruction, chat, and reasoning data. It is suitable for evaluation, post-training, and fine-tuning; the included chat template is a prompt interface, not a claim that the checkpoint is a safety-aligned assistant. The Transformers integration, conversion, release packaging, linear-time RWKV tokenizer, and optional TileLang inference implementation are distributed with this release. • Constant recurrent state: memory does not grow like an attention KV cache. • Bundled Transformers integration: auditable remote configuration and modeling • modules provide generation, recurrent cache continuation, training, and LoRA • workflows on Transformers 5.15+. • Exact linear-time tokenizer: the bundled RWKV trie reads the self-contained • tokenizer.json generated from the canonical RWKV World byte vocabulary. • Chat-ready: chat_template.jinja supports system, multi-turn, thinking, and • strict model-generated tool-call prompts. • Optional optimized runtime: the isolated inference/ bundle • provides PyTorch fallback and TileLang acceleration without changing the • standard model root.

Full model card on Hugging Face →

Alternatives to RWKV7-G1k-13.3B-20260930

All alternatives →
Qwen logo

Qwen3-14B

Qwen·Chat

≈11 GB

Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.

14.8B40K ctx Apache 2.0
13MApr 2025ollama
ornith-ai logo

Ornith-1.5-9B

ornith-ai·Chat

≈7.2 GB

Chirp Chirp! 🐦 We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement.

9.7B256K ctx MIT
570.4KAug 2026ollama
ornith-ai logo

Ornith-1.0-9B

ornith-ai·Chat

≈6.8 GB

Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.

9B256K ctx MIT
822.7KJun 2026gguf
openai logo

gpt-oss-20b

openai·Chat

≈13 GB

Welcome to the gpt-oss series, OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases.

20.9BA4.2B128K ctx Apache 2.0
5.6MAug 2025ollama
speakleash logo

Bielik-11B-v3.0-Instruct

speakleash·Chat

≈8.3 GB

Bielik-11B-v3.0-Instruct is a generative text model featuring 11 billion parameters. It is an instruct fine-tuned version of the Bielik-11B-v3-Base-20250730.

11.2BApache 2.0
99.6KNov 2025gguf
nvidia logo≈9.8 GB

Nemotron-Labs-Diffusion is a tri-mode language model that supports both AR decoding and diffusion-based parallel decoding by simply switching the attention pattern of the same model during inference.

13.5B256K ctx nvidia-nemotron-open-model-license
6.5KApr 2026gguf

RWKV7-G1k-13.3B-20260930 FAQ

How much VRAM does RWKV7-G1k-13.3B-20260930 need?
About 16 GB at Q4_K_M with an 8K context. Q8_0 needs about 22 GB and full precision about 34 GB. Longer contexts need more memory for the KV cache.
Can RWKV7-G1k-13.3B-20260930 run on the Tesla P40?
Yes. It fits at Q8_0 (about 22 GB), at an estimated 14 tokens/s. It fits comfortably on the Radeon RX 7900 XTX, Instinct MI50 32GB, RTX 5090.
Can I use RWKV7-G1k-13.3B-20260930 commercially?
Yes. It is released under Apache 2.0. Permissive: use, modify and sell freely; keep the notice and licence text.
How do I run RWKV7-G1k-13.3B-20260930 locally?
It runs with Transformers. Each has copy-paste examples in its docs.

Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.