# ggrun

> ggrun is a self-hosted terminal UI for discovering, downloading, and running local GGUF models with automatic multi-GPU memory placement and an OpenAI-compatible API.

- **Category:** LLM Inference
- **Licence:** MIT
- **Latest version:** v3.2.12
- **Open source:** oss
- **Languages:** Go, Python, Shell
- **Platforms:** Docker, Docker Compose, Linux, Windows, macOS, Build from Source
- **Tags:** Vulkan, LLM, Inference, Gguf, Cuda, Docker, Golang, Self Hosted, Openai Api, Terminal ui, Multi gpu, Claude code, Llama.cpp, Local llm, Speculative decoding, Moe, Metal, Inference Server, Llama Cpp, Localllama, Ollama Alternative

## Description

ggrun (from "gguf run") is a launcher for running local large language models on your own hardware. It provides a terminal UI for model discovery, downloads from Hugging Face, backend selection (llama.cpp, ik_llama.cpp, Vulkan), and automatic memory placement across GPUs and system RAM — especially for large MoE models. The tool builds a launch plan from the GGUF metadata and your hardware, checks that the plan fits in available VRAM/RAM, starts the inference server, and provides an OpenAI-compatible /v1 API alongside an Anthropic-compatible endpoint for Claude Code workflows. It supports speculative decoding, AI-assisted tuning of launch flags, and measured performance caching. Installation options include a one-line script, native Windows installer, Docker images (CPU, CUDA, Vulkan), and release bundles for Linux, macOS, and Windows.

## Links

- [Homepage](https://github.com/raketenkater/llm-server)
- [GitHub Repository](https://github.com/raketenkater/ggrun)

---
Source: https://selfhost.directory/project/ggrun