Self-host Ollama
Lightweight local model runner and OpenAI-compatible API server for downloading and serving open LLMs on your own hardware, with optional NVIDIA/AMD GPU acceleration — CPU-only also works for smaller models.
Key facts
Reach for Ollama when you want the simplest way to download and serve open LLMs on your own hardware — one command pulls a model and exposes an OpenAI-compatible API on port `11434`. It's the de facto local-model runtime and the easiest thing in this roster to start (difficulty 1). It's a backend, not a chat app: pair it with Open WebUI or LibreChat for a UI. If you also need image or voice models, LocalAI covers more model types.
Follow the Ollama deploy guide →What you need
- Any VPS with at least 8192 MB of RAM
- A domain you control — most self-hosted setups need HTTPS in front of them
- Under an hour if you've used Docker before
Paid link — we earn a commission if you shop through it.
Install
Run these commands on your server:
# Ollama — official CPU container (add --gpus=all for NVIDIA)
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
docker exec -it ollama ollama run llama3.2 # pull + chat with a modelWhat you take on
Ollama is a model server, and treating it like a finished product is where people trip:
An alternative to
Head-to-head
The headless server daemon vs. the polished desktop on-ramp.
The one-command local LLM runtime vs. the Swiss-army model server.
The one-command local runtime vs. the GPU-first production inference engine.
