Self-host litellm
v1.108.9Open-source AI gateway for 100+ LLMs via OpenAI API
- Stars
- 63 307
- Port
- 4000 (HTTP API and Admin UI)
docker.litellm.ai/berriai/litellmWhat is litellm?
LiteLLM provides a single OpenAI-compatible API to access over 100 large language model providers such as OpenAI, Anthropic, Bedrock, Azure, Vertex AI, and Google Gemini. It works as a Python library for direct integration or as a standalone proxy server that adds virtual keys, spend tracking, budgets, rate limits, guardrails, load balancing with automatic fallbacks, caching, and a web-based admin dashboard. The proxy persists model configurations, credentials, and usage logs in PostgreSQL and can use Redis for distributed caching and load-balancer state. Enterprise capabilities like SSO/SAML, audit logs, role-based access control, secret-manager integration, IP allowlists, multi-region deployments, and dedicated support are enabled by a license key on the same container image. The core is MIT-licensed; enterprise features use a commercial license.
Should you self-host litellm?
litellm scores 87/100 on selfhost.directory (how scoring works) and ranks #8 of 123 in AI. It has 63 307 GitHub stars, +2 324 in the last 30 days.
It shipped 129 stable releases in the past year. The latest, v1.108.9, came out on 14 November 2026.
Setup is moderate: expect some configuration beyond a single command. An official Docker image is available. It is MIT-licensed and written mostly in Python.
Its closest alternative on selfhost.directory is openclaw (93/100, 394 525 stars). It scores higher overall, so compare the two before you pick. See litellm vs openclaw side by side.
litellm README
LiteLLM AI Gateway
Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format.
More in AI
Live Helper ChatLive Helper Chat - live support for your website. Featuring web and mobile apps, Voice & Video & ScreenShare. Supports Telegram, Twilio (whatsapp), Facebook messenger including building a bot.
llama.cppllama.cpp is an open-source, self-hosted LLM inference engine written in C/C++ that runs quantized models locally on CPU and GPU with minimal dependencies.
llamactlSelf-hosted control plane that manages llama.cpp, MLX, and vLLM inference servers through a web dashboard with a unified OpenAI-compatible API.
llmakerllmaker is a self-hosted, open-source CLI that provisions and manages complete LLM stacks on your own hardware with a single command.
llm-gatewaySelf-hosted open-source API gateway routing 250+ LLM models from 40+ providers via unified OpenAI-compatible API with cost tracking, caching, and failover.
llm-gatewaySelf-hosted OpenAI-compatible API gateway routing to OpenAI, Anthropic, and local backends with semantic routing, load balancing, and zero-trust networking via zrok.
LLMII (ImageIndexer)Self-hosted desktop app using a local vision LLM to caption and keyword images, embedding results in standard metadata fields for any photo manager.
LLMKubeOpen-source Kubernetes operator running LLMs on NVIDIA, AMD, Intel, and Apple Silicon with pluggable runtimes and hybrid routing.
LLMPID-ASSelf-hosted, open-source API service that detects prompt injections in LLM inputs using a fine-tuned BERT model, with JWT authentication and classification logging.
LLM VisionSelf-hosted Home Assistant integration using multimodal LLMs to analyze camera feeds, images, videos, and Frigate events with intelligent notifications and event timeline
llotSelf-hosted, open-source translation service using local LLMs via Ollama with 38 languages, real-time translation, dark mode, and Wyoming TTS support.
lmgateSelf-hosted gateway adding enterprise auth, rate limits, and model ACLs to Ollama, llama.cpp, and LM Studio via Docker Compose.
0 comments