Aloha! 🌺 Today, we are releasing Ornith-1.0, a self-improving family of open-source models for agentic coding.
UserLM-2.0
microsoft·Instruct·released 1 Oct 2026
llama-server -hf liodon-ai/UserLM-2.0-imatrix-GGUF:Q4_K_MUserLM-2.0 is an 8-billion-parameter user simulator for user-AI conversations. Unlike typical language models trained to play the assistant, UserLM-2.0 generates the user side of a conversation. It is conditioned on a task intent that describes the user's high-level objective.
How much VRAM does UserLM-2.0 need?
At Q4_K_M UserLM-2.0 needs about 6.3 GB with an 8K context (6.3 GB at Q4, 17 GB at full precision). Pick your GPU to see what fits and how fast it runs.
Needs: ≈6.3 GB · Q4_K_M · 8K context — Pick a GPU to check the fit.
Advanced: context 8K · KV cache 16-bit
Smallest GPU for each quant and context (KV cache 16-bit)
| Quantisation | 4K | 8K | 16K | 32K | 64K | 128K |
|---|---|---|---|---|---|---|
| Q3_K_M | ||||||
| Q4_K_M | ||||||
| Q5_K_M | ||||||
| Q6_K | ||||||
| Q8_0 | ||||||
| FP16 |
≤16 GB card24–32 GB48–96 GBmore
| Quantisation | Total |
|---|---|
| Q2_KHeavy lossSmallest, noticeably worse quality | ≈4.1 GB |
| Q3_K_MNoticeable lossSmall, some quality loss | ≈5.3 GB |
| Q4_K_MrecommendedSmall lossRecommended default for local use | ≈6.3 GB |
| Q5_K_MNear-losslessNear-lossless for most tasks | ≈7.1 GB |
| Q6_KNear-losslessPractically lossless | ≈8.0 GB |
| Q8_0LosslessLossless in practice, twice the size of Q4 | ≈9.8 GB |
| FP16 / BF16LosslessFull precision, as released | ≈17 GB |
Estimates for 8K tokens of context: weights + KV cache + runtime overhead. Speed assumes memory-bound decoding at your GPU’s bandwidth. Pick a GPU above and the fit column checks every row against it. Weights marked “file” use the real GGUF download size; “est.” rows are computed from the parameter count.
How to pick a quant
Take the biggest quant that fits with room left for context. Q4_K_M is the usual sweet spot. Below Q4 quality drops fast, above Q6 you mostly pay memory for nothing. Big models (70B+, large MoE) tolerate low quants better than small ones.
- LosslessF16, Q8_0
- Near-losslessQ6_K, Q5_K_M
- Small lossQ5_K_S, Q4_K_M, IQ4_NL, IQ4_XS, Q4_K_S
- Noticeable lossQ4_0, Q3_K_L, Q3_K_M, IQ3_M
- Heavy lossIQ3_XXS, Q3_K_S, IQ2_M, Q2_K
Typical quality loss, not measured per model. Tiers come from KL divergence against FP16 on Llama 3 8B: llama.cpp perplexity README: LLaMA 3 8B scoreboard (KLD and ΔPPL vs FP16, Wikitext-2); llama.cpp quantize tool: ΔPPL per quant type @ Llama-3-8B
How much context fits?
At 8K context, Q4_K_M needs ≈6.3 GB(4.6 GB weights + 1.0 GB KV cache + 0.7 GB overhead). Pick a GPU above to see how much context fits.
Weights + overheadKV cache
Can the RTX 4060 run UserLM-2.0? →
Speed vs quality per quant
Smaller quants run faster and fit in less memory, but lose quality.
Fits in VRAMTight fitCPU offload✕ won't runQuality is typical KL divergence vs FP16, not measured for this model.
Real-world speed of UserLM-2.0
Tokens per second measured by people running it at home. A GPU shows a median once 3 reports agree on the same setup. Reports far from our estimate wait for a review. Measure yours with llama-bench →
No medians yet. Be one of the first 3 to report a setup.
All reports
Run UserLM-2.0 locally
--quantQ4_K_Mdefault; pick your GPU to tailor itliodon-ai/UserLM-2.0-imatrix-GGUF
llama-server -hf liodon-ai/UserLM-2.0-imatrix-GGUF:Q4_K_MDownloads the GGUF on first run. Installed through llama.app? The same flags work with `llama serve`.
vllm serve microsoft/UserLM-2.0 vllm serve liodon-ai/UserLM-2.0-FP8services:
vllm:
image: vllm/vllm-openai:latest
command: microsoft/UserLM-2.0
ports:
- "8000:8000"
ipc: host
volumes:
- ~/.cache/huggingface:/root/.cache/huggingface
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]API on port 8000.
sglang serve microsoft/UserLM-2.0 --host 0.0.0.0 --port 30000services:
sglang:
image: lmsysorg/sglang:latest
command: sglang serve microsoft/UserLM-2.0 --host 0.0.0.0 --port 30000
ports:
- "30000:30000"
ipc: host
volumes:
- ~/.cache/huggingface:/root/.cache/huggingface
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]API on port 30000.
from transformers import pipeline
pipe = pipeline("text-generation", model="microsoft/UserLM-2.0", dtype="auto", device_map="auto")
messages = [{"role": "user", "content": "Explain self-hosting in one paragraph."}]
out = pipe(messages, max_new_tokens=512)
print(out[0]["generated_text"][-1]["content"])Quantised versions and downloads
| Repository | Format | Quant | Size |
|---|---|---|---|
| liodon-ai/UserLM-2.0-FP8 | Safetensors | FP8 | — |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | IQ2_M | 2.9 GB |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | IQ3_M | 3.8 GB |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | IQ4_XS | 4.4 GB |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | Q4_K_M | 4.9 GB |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | Q5_K_M | 5.7 GB |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | Q6_K | 6.6 GB |
| liodon-ai/UserLM-2.0-imatrix-GGUF | GGUF | Q8_0 | 8.5 GB |
| liodon-ai/UserLM-2.0-GGUF | GGUF | — | — |
About UserLM-2.0
UserLM-2.0 is an 8-billion-parameter user simulator for user-AI conversations. Unlike typical language models trained to play the assistant, UserLM-2.0 generates the user side of a conversation. It is conditioned on a task intent that describes the user's high-level objective. UserLM-2.0 can: • Generate an initial user utterance from a task intent. • Generate follow-up user utterances from the conversation so far. • Emit when it determines that the user's objective has • been satisfied and the conversation should end. The model is intended to support research on realistic user simulation and the development and evaluation of more robust assistant models. Paper: The accompanying UserLM-2.0 paper is forthcoming. An arXiv link will be added when it becomes available. • Developed by: Omer Gul (Microsoft Research intern, Spring 2026), • Philippe Laban (Microsoft Research), Tanya Goyal (Microsoft Research), and • Jennifer Neville (Microsoft Research) • Model type: Generative causal language model, 8B parameters • Language: English • License: MIT • Backbone: meta-llama/Llama-3.1-8B-Instruct • Repository: microsoft/UserLM-2.0
Full model card on Hugging Face →Alternatives to UserLM-2.0
All alternatives →Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models.
Chirp Chirp! 🐦 We are introducing Ornith-1.5, a major step toward building foundation models through end-to-end self-improvement.
NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 is a small language model (SLM) further post-trained from Gemma-4-E4B-it by NVIDIA. This is the full-precision (BF16) release.
Granite is a family of open-source large language models developed by IBM, designed for enterprise and research use. Granite models are built to be versatile, safe, and efficient — covering a range of sizes and capabilities from compact edge-deployable models to large-scale reasoning systems.
Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, we release a number of base language models and instruction-tuned language models ranging from 0.5 to 72 billion parameters. Qwen2.5 brings the following improvements upon Qwen2:
UserLM-2.0 FAQ
- How much VRAM does UserLM-2.0 need?
- About 6.3 GB at Q4_K_M with an 8K context. Q8_0 needs about 9.8 GB and full precision about 17 GB. Longer contexts need more memory for the KV cache.
- Can UserLM-2.0 run on the RTX 4060?
- Yes. It fits at Q5_K_M (about 7.1 GB), at an estimated 26 tokens/s. It fits comfortably on the Apple M4 (16 GB), Radeon RX 6700 XT, RTX 3060 12GB.
- Can I use UserLM-2.0 commercially?
- Yes. It is released under MIT. Permissive: do almost anything as long as you keep the copyright notice.
- What is the context length of UserLM-2.0?
- 131,072 tokens (128K). The full window needs far more memory than an 8K context.
- How do I run UserLM-2.0 with llama.cpp?
- Run llama-server -hf liodon-ai/UserLM-2.0-imatrix-GGUF:Q4_K_M. It downloads the GGUF and starts an OpenAI-compatible server.
Rate this model
Data from Hugging Face, the model card and public leaderboards. Memory and speed figures are estimates; real numbers depend on the runtime, context length and drivers. Always check the licence yourself before commercial use.
0 comments