selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

llama.cpp logo

Self-host llama.cpp

v1.0.0

High-performance LLM inference engine in C/C++

ReleaseOpen SourceMIT
Updated2 days ago
LanguageC++
Setup difficultyModerate
Stars
135 257
Ports
9931 (OpenAI-compatible API server (default)), 8080 (Docker container port for llama-server)
Docker
ghcr.io/ggml-org/llama.cpp

What is llama.cpp?

llama.cpp is a high-performance LLM inference engine written in plain C/C++ with zero dependencies. It enables running quantized large language models locally on diverse hardware including Apple Silicon, x86 CPUs with AVX/AVX2/AVX512, NVIDIA GPUs via CUDA, AMD GPUs via HIP, and other accelerators. The project provides a command-line interface (llama-cli), an OpenAI-compatible API server (llama-server), and tools for model conversion and quantization. It supports 1.5-bit through 8-bit integer quantization for reduced memory usage and faster inference. The engine is built on top of the ggml tensor library and offers hybrid CPU+GPU inference for models larger than VRAM. Users can install via Docker, pre-built binaries, package managers, or build from source.

Should you self-host llama.cpp?

llama.cpp scores 93/100 on selfhost.directory (how scoring works) and ranks #2 of 25 in LLM Inference. It has 135 257 GitHub stars, +4 296 in the last 30 days.

It shipped 1742 stable releases in the past year. The latest, b12430, came out on 16 November 2026.

Setup is moderate: expect some configuration beyond a single command. An official Docker image is available. It is MIT-licensed and written mostly in C++.

Its closest alternative on selfhost.directory is Ollama (94/100, 187 151 stars). It scores higher overall, so compare the two before you pick. See llama.cpp vs Ollama side by side.

Features

  • AMD GPU support
  • Apple Silicon optimization
  • Command Line Interface
  • GBNF grammar support
  • Hybrid CPU+GPU inference
  • LLM inference
  • Multiple backend support
  • NVIDIA GPU support
  • OpenAI-compatible API server
  • Quantization Support
  • VLM inference
  • Zero Dependencies

llama.cpp README

llama.cpp releases & version history

@ Every self-hosted release, weekly

One Monday email with the week's stable releases across the directory.

Models llama.cpp runs

Popular open-weight models with llama.cpp support, with VRAM needs and licence at a glance.

openai logo20.9BA4.2BCommercial OK≈13 GB↓ 5.6M
meta-llama logo8BConditions≈6.3 GB↓ 4.2M
openai logo117BA5.8BCommercial OK≈70 GB↓ 2.4M
zai-org logo
GLM-5.3zai-org
753BA41.8BCheck licence≈476 GB↓ 1.8M
Qwen logo8.2BCommercial OK≈6.5 GB↓ 6.1M
Qwen logo752MCommercial OK≈1.8 GB↓ 17.8M
Qwen logo14.8BCommercial OK≈11 GB↓ 13M
LiquidAI logo
LFM3-1.7BLiquidAI
1.7BConditions≈1.9 GB↓ 103.7K

Browse all 1 815 models →

More in LLM Inference

Rate this project

Click to rate — no account needed.

0 comments

Posting as guest
Loading comments…