How to Deploy Ollama on a VPS

copyapt update && apt upgrade -y
adduser deploy
usermod -aG sudo deploy

copyufw allow OpenSSH
ufw allow 80
ufw allow 443
ufw enable

copycurl -fsSL https://get.docker.com | sh
usermod -aG docker deploy

#Install Ollama

The simplest path is the official Docker image. Create a working directory:

copymkdir ~/ollama && cd ~/ollama

Create a compose file:

copyservices:
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    ports:
      - "127.0.0.1:11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]  # comment out if no GPU

volumes:
  ollama_data:

Start it:

copydocker compose up -d

Ollama listens on port 11434 by default. The 127.0.0.1 bind keeps it private — a reverse proxy will handle public HTTPS access.

#GPU support

If you have an NVIDIA GPU, install the NVIDIA Container Toolkit first:

copycurl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
  gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
apt update
apt install -y nvidia-container-toolkit
nvidia-ctk runtime configure --runtime=docker
systemctl restart docker

Without a GPU, just remove the deploy.resources section from the compose file — Ollama falls back to CPU inference automatically.

#Pull your first model

Pull a model and test it:

copydocker exec -it ollama ollama pull llama3.2
docker exec -it ollama ollama run llama3.2

That downloads the model weights (a few GB for 3B/7B variants) and drops you into an interactive chat. Type your message, press Enter, and watch it generate. /bye exits.

To pull a larger model later:

copydocker exec -it ollama ollama pull llama3.1:8b

The API is available immediately — no restart needed. Test it:

copycurl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "What is the capital of France?"
}'

#HTTPS + domain

Ollama's API should not face the internet without TLS. Point a reverse proxy at 127.0.0.1:11434 and terminate HTTPS on 443.

The simplest path is Automatic HTTPS with Caddy. Point an A record for your hostname (say ai.example.com) at the server's public IP, then have Caddy reverse-proxy that hostname to 127.0.0.1:11434. Caddy handles certificate issuance and renewal automatically.

If you're using the Caddy container approach from that guide, put Ollama and Caddy in the same compose file and proxy to the Ollama service name:

reverse_proxy ollama:11434

#Using the API

Once HTTPS is up, point any OpenAI-compatible client at your endpoint. Most tools just need two values:

• Base URL: https://ai.example.com/v1
• Model: any model you've pulled (e.g., llama3.2)

Examples with curl:

copy# Chat completion (OpenAI-compatible)
curl https://ai.example.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.2",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# List available models
curl https://ai.example.com/v1/models

Tools like Open WebUI, LibreChat, and AnythingLLM can all point at this endpoint. If you're building your own app, the OpenAI SDK works with Ollama by changing the base URL — no code changes needed in most cases.

#Backups

Ollama's state is small — just the model weights in the /root/.ollama volume. The weights themselves are re-downloadable, so you don't strictly need to back them up. What matters is your configuration and any custom Modelfiles you've created.

If you've customized models (system prompts, parameters, Modelfiles), back up the data volume:

copydocker compose exec ollama tar czf - /root/.ollama > ollama-config-$(date +%F).tar.gz

For a more complete backup including all downloaded models:

copydocker run --rm -v ollama_data:/data -v $(pwd):/backup alpine \
  tar czf /backup/ollama-full-$(date +%F).tar.gz -C /data .

This is a one-time cost — models don't change once downloaded, so you only need to back up when you add or modify something.

#Upgrades

Pull the newer image and recreate:

copydocker compose pull
docker compose up -d

Ollama updates are usually non-breaking — the API is stable and model formats are backward-compatible. Check the Ollama blog for any notable changes.

#Scaling for production

For heavier usage, consider these additions:

Separate model storage: Mount a larger disk for models if your root volume is small:

copyvolumes:
  - /mnt/models:/root/.ollama

Multiple instances: Run separate Ollama containers for different model sizes or use cases, each on a different port.

Load balancing: For high-concurrency setups, put multiple Ollama instances behind a load balancer. Ollama handles one request per model at a time, so multiple instances means more parallel requests.

Nginx reverse proxy: For finer control than Caddy, use Nginx with rate limiting and caching. See the Next.js deployment guide for Nginx configuration patterns.

#Troubleshooting

Model download fails or hangs. Check your disk space (df -h) — 7B models are 4-8 GB. Also check that you can reach ollama.com from the server (curl -I https://ollama.com).

Out of memory during inference. The model is too large for your RAM. Use a smaller model or a more aggressive quantization (e.g., q4_0 instead of q8_0). Check docker logs ollama for memory errors.

GPU not detected. Verify the NVIDIA Container Toolkit is installed and Docker can see the GPU: docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi. If that fails, re-run the toolkit installation above.

Slow responses on CPU. This is expected — CPU inference is 5-10x slower than GPU. For better performance, use a smaller model or add a GPU. The tokens/second metric in the response tells you the actual throughput.

API returns "model not found." The model name must match exactly what you pulled. Check available models with curl http://localhost:11434/api/tags.

#Verification + next steps

You're done when you can: pull a model, run it interactively, make API calls over HTTPS on your domain, and get coherent responses. The API should work with any OpenAI-compatible client.

From here, pair Ollama with Open WebUI for a ChatGPT-style browser interface, or integrate it into your applications via the API. For document-based AI workflows, AnythingLLM combines Ollama with a document management system. An 8 vCPU / 16 GB box is the right tier for 7B models; a GPU instance is the step up. See Best VPS for AI & ML Workloads for the ranked picks.