How to Deploy Prometheus on a VPS

copysudo ufw allow OpenSSH
sudo ufw allow 80
sudo ufw allow 443
sudo ufw --force enable
sudo ufw status verbose

#Write the configuration

Prometheus reads prometheus.yml at startup. Create the project directory and a config that scrapes Prometheus itself and the node_exporter you're about to run:

copymkdir -p ~/prometheus && cd ~/prometheus
cat > prometheus.yml <<'YAML'
global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: prometheus
    basic_auth:
      username: admin
      password_file: /etc/prometheus/admin-password
    static_configs:
      - targets: ["127.0.0.1:9090"]

  - job_name: node
    static_configs:
      - targets: ["127.0.0.1:9100"]
YAML

The basic_auth block on the prometheus job is there because the next step turns on authentication for Prometheus's own web server. Its self-scrape has to log in like any other client.

Generate the admin password and its bcrypt hash. Prometheus only accepts bcrypt hashes in its web config, never plaintext:

copycd ~/prometheus
sudo apt-get install -y apache2-utils
[ -f admin-password ] || openssl rand -hex 16 > admin-password
chmod 644 admin-password
chmod 700 ~/prometheus
HASH=$(htpasswd -nbBC 10 "" "$(cat admin-password)" | tr -d ':\n')
printf 'basic_auth_users:\n  admin: %s\n' "$HASH" > web.yml
cat web.yml

The password is in ~/prometheus/admin-password. Read it with cat when you first log in. It is readable inside the container because Prometheus runs as the unprivileged nobody user and needs it for the self-scrape. The chmod 700 on the directory keeps other accounts on the server away from it.

#Install Prometheus and node_exporter (Docker Compose)

Both containers use the host network. node_exporter requires it, since it has to see the host's real network interfaces and processes, as its README explains. Running Prometheus the same way lets it reach node_exporter on 127.0.0.1 with no published ports:

copycd ~/prometheus
cat > docker-compose.yml <<'YAML'
services:
  prometheus:
    image: prom/prometheus:v3.15.0
    container_name: prometheus
    restart: unless-stopped
    network_mode: host
    command:
      - --config.file=/etc/prometheus/prometheus.yml
      - --storage.tsdb.path=/prometheus
      - --storage.tsdb.retention.time=30d
      - --web.listen-address=127.0.0.1:9090
      - --web.config.file=/etc/prometheus/web.yml
      - --web.enable-lifecycle
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - ./web.yml:/etc/prometheus/web.yml:ro
      - ./admin-password:/etc/prometheus/admin-password:ro
      - prometheus-data:/prometheus

  node-exporter:
    image: quay.io/prometheus/node-exporter:v1.12.1
    container_name: node-exporter
    restart: unless-stopped
    network_mode: host
    pid: host
    command:
      - --path.rootfs=/host
      - --web.listen-address=127.0.0.1:9100
    volumes:
      - /:/host:ro,rslave

volumes:
  prometheus-data:
YAML

The flags are worth reading once:

• command: replaces the image's defaults entirely. The image's default command is --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.path=/prometheus, so both are repeated here. Leave out --storage.tsdb.path and the data ends up outside the volume, and a container recreate wipes it.
• --storage.tsdb.retention.time=30d sets how long data is kept. Without it (and without --storage.tsdb.retention.size) the default is 15 days. Choose the number on purpose.
• --web.enable-lifecycle lets you reload the config with an HTTP POST instead of a restart.
• --path.rootfs=/host tells node_exporter that the host's filesystem is mounted at /host, so disk metrics describe the server and not the container.

Check the config with promtool (shipped in the same image) before you start anything. A typo here otherwise shows up as a crash loop. The --user flag runs the check as you, because the directory is now private to your account:

copycd ~/prometheus
docker run --rm --user "$(id -u):$(id -g)" --entrypoint promtool \
  -v "$PWD":/etc/prometheus:ro prom/prometheus:v3.15.0 \
  check config /etc/prometheus/prometheus.yml

Start the stack and wait for Prometheus to report ready:

copycd ~/prometheus
docker compose up -d
for i in $(seq 1 30); do
  curl -fs -u "admin:$(cat admin-password)" http://127.0.0.1:9090/-/ready && break
  sleep 2
done
docker compose ps

Confirm authentication is enforced and both targets are being scraped. The first curl has no credentials and should print 401:

copycd ~/prometheus
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:9090/api/v1/targets
sleep 20
curl -s -u "admin:$(cat admin-password)" http://127.0.0.1:9090/api/v1/targets \
  | grep -o '"health":"[a-z]*"'

Two "health":"up" lines mean both jobs are healthy. If node shows down, node_exporter isn't running. Check docker compose logs node-exporter.

#HTTPS + domain

Point an A record for metrics.example.com at the server, then put Caddy in front of the loopback listener:

metrics.example.com {
    reverse_proxy 127.0.0.1:9090
}

Caddy installed on the host reaches 127.0.0.1:9090 directly. If you run Caddy as a container, 127.0.0.1 is the container's own loopback, so give that container network_mode: host as well.

The browser now asks for the admin user and the password from ~/prometheus/admin-password. Basic auth sits in Prometheus itself, not in Caddy, so it protects the API even for something on the box that bypasses the proxy.

#Add a target and reload

Adding a target means editing prometheus.yml. For example, a second server running its own node_exporter:

copy  - job_name: node-web1
    static_configs:
      - targets: ["10.0.0.12:9100"]

Exporters on other machines have to be reachable from this box and should not be reachable from anywhere else. Use a private network or a WireGuard tunnel, or firewall port 9100 to this server's IP only. Then validate and reload without a restart:

copycd ~/prometheus
docker run --rm --user "$(id -u):$(id -g)" --entrypoint promtool \
  -v "$PWD":/etc/prometheus:ro prom/prometheus:v3.15.0 \
  check config /etc/prometheus/prometheus.yml
curl -s -X POST -u "admin:$(cat admin-password)" http://127.0.0.1:9090/-/reload

#Backups

The config files are small and precious. The TSDB is large and replaceable, since losing it costs you history and not the setup. Back up both, and stop the container for a consistent copy of the data:

copycd ~/prometheus
tar czf prometheus-config-$(date +%F).tar.gz prometheus.yml web.yml docker-compose.yml
docker compose stop prometheus
docker run --rm -v prometheus_prometheus-data:/data -v "$PWD":/backup alpine \
  tar czf /backup/prometheus-data-$(date +%F).tar.gz -C /data .
docker compose start prometheus
ls -lh ~/prometheus/*.tar.gz

The volume name is prefixed with the compose project directory (prometheus_). Confirm it with docker volume ls. Copy the archives off the box. The password file is left out on purpose. Store it in your password manager instead.

#Upgrades

Change the pinned tags in docker-compose.yml to the new releases, then:

copycd ~/prometheus
docker compose pull
docker compose up -d

Read the release notes before a major version jump. Prometheus 3 changed the UI and some flag defaults compared with 2.x. The TSDB format is carried forward, but take a backup first.

#Troubleshooting

The container restarts in a loop. Run docker compose logs prometheus. The usual causes are a YAML error (run promtool check config), a prometheus.yml that Docker created as a directory because the file didn't exist at first start (delete it, write the file, recreate the container), or a malformed web.yml.

Every request returns 401, even with the right password. The hash in web.yml must be bcrypt. Regenerate it with htpasswd -nbBC 10 as above. Plaintext and MD5 hashes are rejected.

The prometheus target is down with a 401. The self-scrape needs the basic_auth block in prometheus.yml. The password file must be mounted and readable by the nobody user inside the container.

Disk keeps growing. Check the retention flags and the series count (prometheus_tsdb_head_series in the query box). A label with unbounded values, such as user IDs or full URLs, is the classic cause.

#Verification + next steps

You're done when https://metrics.example.com prompts for a password, the target health page (under Status) shows both jobs up, and the query node_memory_MemAvailable_bytes returns a value. Then restart the stack and confirm the history is still there.

Next: add Grafana and point a Prometheus data source at http://127.0.0.1:9090 with the same basic-auth credentials. For a lighter all-in-one host monitor, see Netdata. If you want logs and traces in the same place as your metrics, see SigNoz.