Self-host voidllm

VoidLLM runs on your own infrastructure as a gateway for LLM traffic. Applications send OpenAI-compatible requests to it, and it forwards them to whichever provider you configure — managed services like OpenAI or Anthropic, or self-hosted options such as Ollama and vLLM. You define short model names that clients use, while the proxy decides where each request actually goes. Four load-balancing methods spread traffic across multiple deployments and automatically retry on failures. Permissions follow a four-tier hierarchy with four roles, and quotas for requests and tokens are enforced at every tier with the tightest limit taking effect. A web console gives you a dashboard, a playground for testing, key administration, and usage reports. An MCP gateway includes six built-in tools and can also forward calls to external MCP servers with per-organization or per-team access rules. Code Mode allows agents to execute sandboxed JavaScript that chains several tool calls together. Metrics are available in Prometheus format. The program is a single Go binary of roughly 25 megabytes; it uses SQLite by default and can switch to PostgreSQL. Installation options include Docker, Docker Compose, Helm, pre-built binaries for major platforms, and a one-click Railway button. Prompts and responses are never persisted — only metadata like token counts and latency is stored. The license is BSL 1.1, which becomes Apache 2.0 after four years. The free tier has all core features; paid tiers add unlimited organizations, cost reporting, SSO, audit logs, distributed tracing, and multi-instance support.