# SmarterRouter

> Self-hosted AI gateway that automatically routes queries to the best local or cloud LLM based on prompt analysis, with VRAM management and OpenAI-compatible API.

- **Category:** AI Chat
- **Licence:** MIT
- **Latest version:** 2.2.5
- **Open source:** oss
- **Languages:** Python
- **Platforms:** Docker Compose, Python
- **Tags:** LLM, Ollama, Ai Gateway, Docker, Fastapi, Self Hosted, Self Hosted Ai, Gpu Monitoring, Model serving, Llama.cpp, Local llm, Openai compatible, Llm proxy, Llm router, Openai proxy, Semantic cache, Ollama Api, Ai Cache, Vram Management, Model Profiling, Semantic Caching

## Description

SmarterRouter is a self-hosted AI gateway that automatically directs each prompt to the most suitable language model. It connects to local backends such as Ollama and llama.cpp as well as cloud APIs from OpenAI, Anthropic, Google, Cohere, and Mistral. The router discovers models, benchmarks them on your hardware, and learns which model handles each task type most effectively. It manages GPU memory across multiple vendors, unloading models when memory runs low. Responses are cached for instant reuse, and failed requests automatically fall back to alternative models. An OpenAI-compatible API makes it a drop-in replacement for existing clients. Admin endpoints expose model profiles, VRAM metrics, cache controls, and a dead-letter queue for failed background jobs, all protected by an API key. Configuration uses environment variables with an interactive setup wizard. Deployment options include Docker Compose with GPU-specific templates, a one-line docker-run.sh script, and manual Python installation.

## Links

- [Homepage](https://github.com/peva3/SmarterRouter)
- [Docker](ghcr.io/peva3/smarterrouter:latest)
- [GitHub Repository](https://github.com/peva3/SmarterRouter)
- [GitHub Issues](https://github.com/peva3/SmarterRouter/issues)
- [Installation Guide](https://github.com/peva3/SmarterRouter/blob/main/docs/installation.md)
- [Configuration Reference](https://github.com/peva3/SmarterRouter/blob/main/docs/configuration.md)
- [API Documentation](https://github.com/peva3/SmarterRouter/blob/main/docs/api.md)
- [OpenWebUI Integration Guide](https://github.com/peva3/SmarterRouter/blob/main/docs/examples/openwebui-integration.md)
- [Production Deployment Guide](https://github.com/peva3/SmarterRouter/blob/main/docs/examples/production-deployment.md)
- [Client Integration Guide](https://github.com/peva3/SmarterRouter/blob/main/docs/examples/client-integration.md)

---
Source: https://selfhost.directory/project/smarterrouter