What is ggrun?

ggrun (from "gguf run") is a launcher for running local large language models on your own hardware. It provides a terminal UI for model discovery, downloads from Hugging Face, backend selection (llama.cpp, ik_llama.cpp, Vulkan), and automatic memory placement across GPUs and system RAM — especially for large MoE models. The tool builds a launch plan from the GGUF metadata and your hardware, checks that the plan fits in available VRAM/RAM, starts the inference server, and provides an OpenAI-compatible /v1 API alongside an Anthropic-compatible endpoint for Claude Code workflows. It supports speculative decoding, AI-assisted tuning of launch flags, and measured performance caching. Installation options include a one-line script, native Windows installer, Docker images (CPU, CUDA, Vulkan), and release bundles for Linux, macOS, and Windows.

Should you self-host ggrun?

ggrun scores 50/100 on selfhost.directory (how scoring works) and ranks #18 of 25 in LLM Inference. It has 282 GitHub stars.

It shipped 11 stable releases in the past year. The latest, v3.2.12, came out on 18 October 2026. New versions land roughly every 2 weeks.

Setup is moderate: expect some configuration beyond a single command. It is MIT-licensed and written mostly in Go.

Its closest alternative on selfhost.directory is AnythingLLM (88/100, 67 666 stars). It scores higher overall, so compare the two before you pick. See ggrun vs AnythingLLM side by side.