# voidllm

> Self-hosted LLM proxy routing to OpenAI, Anthropic, Azure, Ollama, vLLM with load balancing, RBAC, rate limits, and zero-knowledge architecture.

- **Category:** API Gateway
- **Licence:** Other
- **Latest version:** v0.0.30
- **Open source:** source_available
- **Languages:** Typescript, Go
- **Platforms:** Docker, Docker Compose, Windows, Helm, Railway, Build from Source
- **Tags:** Privacy, Privacy First, LLM, Anthropic, Openai, Ollama, Rbac, Zero Knowledge, Api Gateway, Ai Gateway, Rate Limiting, Kubernetes, Docker, AI, Golang, Helm, Self Hosted, Load Balancing, Vllm, Usage tracking, Llm gateway, Llm proxy, Openai proxy, Mcp gateway, Model Routing, Anthropic Proxy, Token Budgets, Code Mode

## Description

VoidLLM runs on your own infrastructure as a gateway for LLM traffic. Applications send OpenAI-compatible requests to it, and it forwards them to whichever provider you configure — managed services like OpenAI or Anthropic, or self-hosted options such as Ollama and vLLM. You define short model names that clients use, while the proxy decides where each request actually goes. Four load-balancing methods spread traffic across multiple deployments and automatically retry on failures. Permissions follow a four-tier hierarchy with four roles, and quotas for requests and tokens are enforced at every tier with the tightest limit taking effect. A web console gives you a dashboard, a playground for testing, key administration, and usage reports. An MCP gateway includes six built-in tools and can also forward calls to external MCP servers with per-organization or per-team access rules. Code Mode allows agents to execute sandboxed JavaScript that chains several tool calls together. Metrics are available in Prometheus format. The program is a single Go binary of roughly 25 megabytes; it uses SQLite by default and can switch to PostgreSQL. Installation options include Docker, Docker Compose, Helm, pre-built binaries for major platforms, and a one-click Railway button. Prompts and responses are never persisted — only metadata like token counts and latency is stored. The license is BSL 1.1, which becomes Apache 2.0 after four years. The free tier has all core features; paid tiers add unlimited organizations, cost reporting, SSO, audit logs, distributed tracing, and multi-instance support.

## Links

- [Homepage](https://github.com/voidmind-io/voidllm)
- [Website](https://voidllm.ai)
- [Docker](ghcr.io/voidmind-io/voidllm)
- [GitHub](https://github.com/voidmind-io/voidllm)
- [Documentation](https://voidllm.ai/docs/)
- [Blog](https://voidllm.ai/blog/)
- [FAQ](https://voidllm.ai/faq/)
- [Changelog](https://github.com/voidmind-io/voidllm/blob/main/CHANGELOG.md)

---
Source: https://selfhost.directory/project/voidllm