selfhost.directory

The index of self-hostable projects — ranked, categorized and served with live updates.
Own your data and stop giving it to big tech.

Ollama logo

Self-host Ollama

v0.40.16

Local-first platform for running open AI models

ReleaseOpen SourceMIT
Updated6 days ago
LanguageGo
Setup difficultyEasy
Stars
187 003
Rating
5.0 (2)
Port
11434 (REST API server)
Docker
ollama/ollama
Install with Docker

What is Ollama?

Ollama lets developers run open-weight LLMs like Llama, Gemma, Qwen, and DeepSeek on their own hardware. It packages model weights, configuration, and the llama.cpp inference engine into a single binary with a REST API compatible with OpenAI and Anthropic clients. Models are pulled from a public library and stored locally. The server supports GPU acceleration on NVIDIA (CUDA), AMD (ROCm), Apple (Metal), and Vulkan backends. It handles concurrent requests, model loading/unloading, context window sizing, and streaming responses. Official SDKs exist for Python and JavaScript, and a large ecosystem of community integrations connects Ollama to IDEs, chat UIs, RAG frameworks, and agent platforms. A cloud tier offers hosted models with usage-based pricing, but local inference remains free and unlimited.

Should you self-host Ollama?

Ollama scores 94/100 on selfhost.directory (how scoring works) and ranks #1 of 25 in LLM Inference. It has 187 003 GitHub stars, +4 211 in the last 30 days.

It shipped 54 stable releases in the past year. The latest, v0.40.16, came out on 8 November 2026. New versions land roughly every 2 days.

Setup is easy: most people get it running in one sitting. An official Docker image is available. It is MIT-licensed and written mostly in Go.

Its closest alternative on selfhost.directory is llama.cpp (93/100, 135 116 stars). Ollama scores higher overall. See Ollama vs llama.cpp side by side.

Features

  • Cloud modelsRequires paid plan for full access; local models always free
  • Concurrent model loading
  • Embeddings
  • GPU Acceleration
  • JavaScript SDK
  • Model import
  • Model library
  • Python SDK
  • REST API
  • Streaming Responses
  • Structured outputs
  • Tool Calling
  • Vision models

Ollama README

How to install Ollama

Install Ollama with Docker

Docker Engine; nvidia-container-toolkit for GPU
Environment variables
OLLAMA_HOST, OLLAMA_MODELS, OLLAMA_CONTEXT_LENGTH, etc.

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

CPU only; use --gpus=all for NVIDIA, ollama/ollama:rocm for AMD

How to configure Ollama

  • OLLAMA_HOST127.0.0.1:11434Bind address for the Ollama server (default: 127.0.0.1:11434)

    values: IP:PORT format

  • OLLAMA_MODELS~/.ollama/models (macOS), /usr/share/ollama/.ollama/models (Linux), C:\Users\%username%\.ollama\models (Windows)Directory where models are stored

    values: Any valid directory path

  • OLLAMA_CONTEXT_LENGTH4096Default context window size in tokens

    values: Integer token count

  • OLLAMA_KEEP_ALIVE5mHow long models stay loaded in memory after last request

    values: Duration string (10m, 24h), seconds (3600), negative to keep forever (-1), 0 to unload immediately

  • OLLAMA_MAX_LOADED_MODELS3 * number of GPUs or 3 for CPUMaximum number of models loaded concurrently

    values: Integer

Ollama releases & version history

@ Every self-hosted release, weekly

One Monday email with the week's stable releases across the directory.

FAQ

How does Ollama handle concurrent requests?

Multiple models can load if memory permits; parallel requests per model scale context and memory. Configure with OLLAMA_MAX_LOADED_MODELS, OLLAMA_NUM_PARALLEL, OLLAMA_MAX_QUEUE.

How do I keep a model loaded in memory?

Use keep_alive parameter: -1 to keep forever, duration like '10m', or set OLLAMA_KEEP_ALIVE env var. Default is 5 minutes.

Where are models stored?

macOS: ~/.ollama/models; Linux: /usr/share/ollama/.ollama/models; Windows: C:\Users\%username%\.ollama\models

How can I expose Ollama on my network?

Change the bind address with OLLAMA_HOST environment variable (default 127.0.0.1:11434).

How do I disable Ollama Cloud features?

Set disable_ollama_cloud: true in ~/.ollama/server.json or set OLLAMA_NO_CLOUD=1 environment variable.

Does Ollama send my prompts and answers back to ollama.com?

No, local runs keep data private. Cloud models process prompts but don't store or train on them.

How do I configure Ollama server?

Configure via environment variables (OLLAMA_HOST, OLLAMA_MODELS, etc.) using launchctl on macOS, systemctl on Linux, or Windows Environment Variables.

How can I tell if my model was loaded onto the GPU?

Run 'ollama ps' and check the Processor column: 100% GPU, 100% CPU, or split percentage.

How can I specify the context window size?

Set OLLAMA_CONTEXT_LENGTH environment variable (default 4096), use /set parameter num_ctx in CLI, or num_ctx in API options.

How can I view the logs?

Refer to the Troubleshooting documentation for log access instructions.

How can I upgrade Ollama?

On macOS and Windows, Ollama auto-downloads updates; click the menu/taskbar item and choose "Restart to update". On Linux, re-run the install script.

Models Ollama runs

Popular open-weight models with Ollama support, with VRAM needs and licence at a glance.

openai logo20.9BA4.2BCommercial OK≈13 GB↓ 5.6M
meta-llama logo8BConditions≈6.3 GB↓ 4.2M
openai logo117BA5.8BCommercial OK≈70 GB↓ 2.4M
zai-org logo
GLM-5.3zai-org
753BA41.8BCheck licence≈476 GB↓ 1.8M
Qwen logo8.2BCommercial OK≈6.5 GB↓ 6.1M
Qwen logo752MCommercial OK≈1.8 GB↓ 18.5M
Qwen logo14.8BCommercial OK≈11 GB↓ 13M
LiquidAI logo
LFM3-1.7BLiquidAI
1.7BConditions≈1.9 GB↓ 100.6K

Browse all 623 models →

More in LLM Inference

Rate this project

Click to rate — no account needed.

0 comments

Posting as guest
Loading comments…