selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

sglang logo

Self-host sglang

v0.5.23

High-performance LLM and multimodal inference framework

ReleaseOpen SourceApache-2.0
Updatedlast week
LanguagePython
Setup difficultyAdvanced
Stars
38 926
Port
30000 (HTTP API server)
Docker
lmsysorg/sglang
Install with Docker

What is sglang?

SGLang is a high-performance serving framework for large language models, vision-language models, and diffusion models. It targets production deployments from a single GPU to distributed clusters, delivering low-latency and high-throughput inference. The framework supports many open models including Llama, Qwen, DeepSeek, GLM, and Mistral, and runs on diverse hardware such as NVIDIA GPUs, AMD GPUs, Google TPUs, Intel CPUs/GPUs, Apple Silicon, Huawei Ascend NPUs, and Moore Threads GPUs. SGLang provides OpenAI-compatible API endpoints, advanced optimizations like disaggregated prefill/decode, speculative decoding, tensor/expert/data parallelism, prefix caching, and optimized GPU kernels. Deployment options include Docker or Python installation with uv. The project is hosted by LMSYS, a non-profit open-source organization, and is licensed under Apache 2.0.

sglang alternatives

Should you self-host sglang?

sglang scores 85/100 on selfhost.directory (how scoring works) and ranks #4 of 25 in LLM Inference. It has 38 926 GitHub stars, +1 874 in the last 30 days.

It shipped 18 stable releases in the past year. The latest, v0.5.23, came out on 30 October 2026. New versions land roughly every 2 weeks.

Setup is advanced: plan for several services and some ongoing maintenance. An official Docker image is available. It is Apache-2.0-licensed and written mostly in Python.

Its closest alternative on selfhost.directory is Ollama (94/100, 187 151 stars). It scores higher overall, so compare the two before you pick. See sglang vs Ollama side by side.

Features

  • Broad hardware support
  • Context parallelism
  • Data parallel attention
  • Disaggregated prefill/decode
  • Expert parallelism
  • LoRA support
  • Multi-modal support
  • OpenAI Compatible API
  • Prefix caching
  • Quantization Support
  • Speculative Decoding
  • Tensor Parallelism

sglang README

SGLang: Fast inference for LLMs and multimodal models

SGLang

PyPI version License: Apache 2.0 PyPI downloads per month

Docs | Cookbook | Website | Blog | Slack

SGLang is an open-source inference framework for large language, vision-language, and diffusion models, optimized for agentic workloads, large-scale serving, and RL rollouts. SGLang Diffusion is its built-in image and video generation engine, included in this repository and the sglang Python package.

How to install sglang

Install sglang with Docker

docker pull lmsysorg/sglang:latest docker run --rm --gpus all --ipc=host
-p 30000:30000
lmsysorg/sglang:latest
sglang serve MODEL_PATH --host 0.0.0.0 --port 30000

Requires NVIDIA Container Toolkit and CUDA 13-compatible driver

sglang releases & version history

@ Every self-hosted release, weekly

One Monday email with the week's stable releases across the directory.

Models sglang runs

Popular open-weight models with sglang support, with VRAM needs and licence at a glance.

openai logo20.9BA4.2BCommercial OK≈13 GB↓ 5.6M
meta-llama logo8BConditions≈6.3 GB↓ 4.2M
openai logo117BA5.8BCommercial OK≈70 GB↓ 2.4M
zai-org logo
GLM-5.3zai-org
753BA41.8BCheck licence≈476 GB↓ 1.8M
Qwen logo8.2BCommercial OK≈6.5 GB↓ 6.1M
Qwen logo752MCommercial OK≈1.8 GB↓ 17.8M
Qwen logo14.8BCommercial OK≈11 GB↓ 13M
Qwen logo7.6BCommercial OK≈5.5 GB↓ 6.5M

Browse all 1 703 models →

More in LLM Inference

Rate this project

Click to rate — no account needed.

0 comments

Posting as guest
Loading comments…