What is llms.py?

llms.py is a lightweight LLM gateway you run on your own hardware: it acts as a command-line tool, an OpenAI-compatible HTTP server and a ChatGPT-style browser UI sitting in front of many model providers. Requests are routed to more than 530 models across 24 providers (definitions come from models.dev), with automatic failover and cost-ordered routing, while local back-ends such as Ollama and LM Studio work without keys. The web app adds project workspaces with a file explorer and Git diff review, agent profiles, outbound MCP tool connections, PDF Studio, Decision Studio, Gemini-powered RAG file stores, microphone voice input, image and speech generation, handcrafted themes with dark mode, and cost/token analytics. A credentials extension provides sign-in, registration, roles and account locking for multi-user installs, with GitHub OAuth as an alternative; data is kept locally. Operator options are the llms-py pip package, the ghcr.io container image via Docker or Compose, a one-line install script that wraps Docker, or native desktop builds for macOS, Windows and Linux.

Should you self-host llms.py?

llms.py scores 50/100 on selfhost.directory (how scoring works) and ranks #28 of 69 in AI Chat. It has 628 GitHub stars.

It shipped 10 stable releases in the past year. The latest, v4.0.26, came out on 20 October 2026. New versions land roughly every 1 days.

Setup is easy: most people get it running in one sitting. An official Docker image is available. It is BSD-3-Clause-licensed and written mostly in JavaScript.

Its closest alternative on selfhost.directory is OmniRoute (88/100, 82 065 stars). It scores higher overall, so compare the two before you pick. See llms.py vs OmniRoute side by side.

Features

Agent profiles
Automatic provider failover
ChatGPT-like Web UI
CLI Tool
Cost and usage analytics
Gemini RAG file stores
Git and workspace explorer
Image and audio generation
Local model support
MCP client connections
Multi-provider LLM routing
Multi-user Authentication
OpenAI Compatible API
PDF Studio
Tool / function calling

How to configure llms.py

ANTHROPIC_API_KEYAnthropic — https://docs.anthropic.com/en/docs/about-claude/models
Anthropic — https://docs.anthropic.com/en/docs/about-claude/models
CEREBRAS_API_KEYCerebras — https://inference-docs.cerebras.ai/models/overview
Cerebras — https://inference-docs.cerebras.ai/models/overview
CHUTES_API_KEYChutes — https://llm.chutes.ai/v1/models
Chutes — https://llm.chutes.ai/v1/models
CODESTRAL_API_KEYcodestral
codestral
DASHSCOPE_API_KEYAlibaba — https://www.alibabacloud.com/help/en/model-studio/models
Alibaba — https://www.alibabacloud.com/help/en/model-studio/models
DEEPSEEK_API_KEYDeepSeek — https://api-docs.deepseek.com/quick_start/pricing
DeepSeek — https://api-docs.deepseek.com/quick_start/pricing
FIREWORKS_API_KEYFireworks AI — https://fireworks.ai/docs/
Fireworks AI — https://fireworks.ai/docs/
GEMINI_API_KEY
GITHUB_TOKENGitHub Copilot — https://docs.github.com/en/copilot
GitHub Copilot — https://docs.github.com/en/copilot
GOOGLE_API_KEYGoogle — https://ai.google.dev/gemini-api/docs/models
Google — https://ai.google.dev/gemini-api/docs/models
GOOGLE_GENERATIVE_AI_API_KEY
GROQ_API_KEYGroq — https://console.groq.com/docs/models
Groq — https://console.groq.com/docs/models
HF_TOKENHugging Face — https://huggingface.co/docs/inference-providers
Hugging Face — https://huggingface.co/docs/inference-providers
LLMSPY_API_KEYllmspy
llmspy
LMSTUDIO_API_KEYLMStudio — https://lmstudio.ai/models
LMStudio — https://lmstudio.ai/models
MINIMAX_API_KEYMiniMax (minimax.io) — https://platform.minimax.io/docs/guides/quickstart
MiniMax (minimax.io) — https://platform.minimax.io/docs/guides/quickstart
MISTRAL_API_KEYMistral — https://docs.mistral.ai/getting-started/models/
Mistral — https://docs.mistral.ai/getting-started/models/
MOONSHOT_API_KEYMoonshot AI — https://platform.moonshot.ai/docs/api/chat
Moonshot AI — https://platform.moonshot.ai/docs/api/chat
NVIDIA_API_KEYNvidia — https://docs.api.nvidia.com/nim/
Nvidia — https://docs.api.nvidia.com/nim/
OLLAMA_API_KEYOllama Cloud — https://docs.ollama.com/cloud
Ollama Cloud — https://docs.ollama.com/cloud
OPENAI_API_KEYOpenAI — https://platform.openai.com/docs/models
OpenAI — https://platform.openai.com/docs/models
OPENAI_LOCAL_API_KEYopenai-local
openai-local
OPENROUTER_API_KEYOpenRouter — https://openrouter.ai/models
OpenRouter — https://openrouter.ai/models
PGID
PUIDDocker Compose user (optional; defaults to 1000:1000). For Linux bind mounts, use the directory owner's `id -u` and `id -g` values. Pre-create the host directory and ensure this user can write to it.
Docker Compose user (optional; defaults to 1000:1000). For Linux bind mounts, use the directory owner's `id -u` and `id -g` values. Pre-create the host directory and ensure this user can write to it.
XAI_API_KEYxAI — https://docs.x.ai/docs/models
xAI — https://docs.x.ai/docs/models
ZHIPU_API_KEYZ.AI / Z.AI Coding Plan — https://docs.z.ai/guides/overview/pricing
Z.AI / Z.AI Coding Plan — https://docs.z.ai/guides/overview/pricing