# oss-llmops-stack

> An open-source, self-hosted LLMOps platform pairing LiteLLM's unified model gateway with Langfuse's tracing, evaluation, and prompt tooling for production AI workloads.

- **Category:** LLM Integration
- **Licence:** MIT
- **Open source:** oss
- **Platforms:** Docker Compose, Helm
- **Tags:** Monitoring, Gateway, Llm Observability, Llm Evaluation, LLMOps, LLM, Ai Gateway, Open Source, Docker Compose, Helm, Self Hosted, Prompt Management, Langfuse, Litellm, Openai compatible, Llm gateway, Openai proxy, Ycombinator

## Description

The OSS LLMOps Stack is a reference architecture that combines two established open-source projects: LiteLLM, which acts as a unified LLM proxy/gateway providing OpenAI-compatible access to 100+ models with failover, load balancing, and cost controls; and Langfuse, which provides observability, prompt management, evaluation, and experimentation capabilities. The stack is deployed via Docker Compose or Helm charts, allowing teams to run the entire LLMOps infrastructure on their own hardware. It integrates LiteLLM callbacks with Langfuse for automatic tracing of LLM requests. The project is maintained by the Langfuse organization and is used in production by companies like Lemonade, Fletch, and Decisional.

## Links

- [Homepage](https://github.com/langfuse/oss-llmops-stack)
- [Website](https://oss-llmops-stack.com)
- [GitHub Repository](https://github.com/langfuse/oss-llmops-stack)
- [Documentation](https://oss-llmops-stack.com/docs)

---
Source: https://selfhost.directory/project/oss-llmops-stack