Self-host libre-chat
0.0.6Self-hosted LLM chatbot with web UI and API
- Stars
- 172
- Port
- 8000 (HTTP API and web UI)
ghcr.io/vemonet/libre-chat
What is libre-chat?
Libre Chat is a free and open-source chatbot web service that lets you run large language models locally on your own hardware. It provides both a web UI and a REST/websocket API for interacting with models like Mixtral or Llama 2 in GGUF format via llama.cpp. The service supports generic conversation and experimental document-based question answering with vector search over uploaded documents (PDF, CSV, HTML, JSON, markdown). It runs fully offline with no API keys required, works on CPU (though slower), and can be deployed via Docker, docker-compose, or pip. Configuration is done through a YAML file. The project is in early development (pre-1.0) and the author recommends more mature alternatives like Ollama or GPT4All for production use.
Should you self-host libre-chat?
libre-chat scores 38/100 on selfhost.directory (how scoring works) and ranks #57 of 69 in AI Chat. It has 172 GitHub stars.
There was no stable release in the past 12 months, so check the repository's recent activity before you rely on it.
Setup is moderate: expect some configuration beyond a single command. An official Docker image is available. It is MIT-licensed and written mostly in Python.
Its closest alternative on selfhost.directory is LibreChat (85/100, 46 414 stars). It scores higher overall, so compare the two before you pick. See libre-chat vs LibreChat side by side.
Features
- CPU-only operationSlower on CPU
- Document-based QA (RAG)Experimental feature
- Gradio alternative UI
- Local LLM Inference
- Multiple model support
- OpenAPI documentation
- REST and WebSocket API
- Vector store with FAISS
- Web UI with streaming
- YAML Configuration
libre-chat README
More in LLM Inference
llama.cppllama.cpp is an open-source, self-hosted LLM inference engine written in C/C++ that runs quantized models locally on CPU and GPU with minimal dependencies.
llamactlSelf-hosted control plane that manages llama.cpp, MLX, and vLLM inference servers through a web dashboard with a unified OpenAI-compatible API.
llama-gptSelf-hosted, open-source ChatGPT-style chatbot that runs Llama 2 and Code Llama locally, with a web chat UI and an OpenAI-compatible API.
LLMKubeOpen-source Kubernetes operator running LLMs on NVIDIA, AMD, Intel, and Apple Silicon with pluggable runtimes and hybrid routing.
LumabriLumabri is a self-hosted, open-source system for running LLMs across LAN computers by distributing layers with donor approval, offering terminal and web chat.
mistral.rsmistral.rs is a self-hosted, open-source LLM inference engine in Rust serving OpenAI-compatible and Anthropic-compatible APIs with built-in web UI.
OllamaOllama is a self-hosted, open-source platform for running LLMs locally on macOS, Windows, and Linux with a REST API and official SDKs.
ollama-intel-gpuA self-hosted Docker image running Ollama on Intel Arc GPUs via IPEX-LLM, drop-in replacement for Nvidia Ollama with same API on port 11434.
production-stackSelf-hosted, open-source Helm stack that scales vLLM across a Kubernetes cluster, with KV-cache-aware request routing and a Prometheus/Grafana monitoring dashboard.
ReameSelf-hosted LLM server for CPU inference using ternary models, runs 8B models in 2.2GB RAM on free-tier ARM with OpenAI-compatible API.
Self-Hosted AI StackOpen-source Docker Compose project running local AI models, chat, documents, voice, and MCP tools on your server with optional HTTPS and GPU acceleration.
sglangSGLang is an open-source, self-hosted inference framework for LLMs and multimodal models with OpenAI-compatible API.
0 comments