Why self-hosted installs fail: 29 of 111 apps did not pass unattended

data as of September 29, 2026

When we run an app's documented install on a clean server, most installs just work. The ones that do not are more useful than the ones that do: each failure is a place where a reader following the same page would also get stuck. This report publishes those failures as data: which apps failed, which category they sit in, and why, as read from the transcript rather than from the exit code.

It is a failure-side companion to the install-quality report and covers the most recent sweeps in the repo. It reports what the harness saw. It says nothing about how often installs fail in general: the apps were chosen by us, not sampled.

The numbers

111 apps, 2026-09-28 sweepeach install attempted on a clean 8 GB Ubuntu 26.04 box. Counts are after later re-runs superseded earlier verdicts.

29 failed first passof 111. 4 passed on a re-run: KeeWeb (our box: native services from an earlier app held ports 80/443); Nhost (our snippet fixed: upstream's pinned MinIO image no longer pulls); NocoDB (our box: native services from an earlier app held ports 80/443); Wiki.js (our box: native services from an earlier app held ports 80/443). Separately, 4 apps that passed the first pass were later counted as failures (see Method), so the total stays 29.

29 did not pass unattendedafter re-runs. Each has a hand-read cause, in these groups: Needs an interactive terminal (11); Needs a hand-edited file first (8); Runs in the foreground (6); Needs a real domain or TLS (1); Too big for the test box (1); Our snippet was wrong (1); Cause not established (1).

3 of 100 failed once, 2026-09-29two further sweeps of other catalog apps; 3 had a failing attempt and 0 still failed at the end.

17 upstream candidatespossible defects in upstream projects noticed along the way; 17 not yet filed or confirmed by the maintainers.

noteHow to read this. Observed: the install did not complete, per the harness transcript. Classified: the cause category was read from that transcript by a person. Candidate: a suspected upstream defect we have not yet reported or had confirmed. None of these is a verdict on a project's quality.

Method

The harness is scripts/verify-install.ts, run by scripts/sweep-installs.sh against a throwaway cloud VM that is pruned between apps. It runs the app's own install commands exactly as written, with a hard timeout; any non-zero exit, or a container left restarting or exited, is a failure. The first-pass verdict column alone is not trusted: every failing transcript was read before it was given a cause.

4 apps exited 0 on every step but are counted as failures here, because a container was down at measurement time, the install needed a hand edit the runner skipped, or the server ran in the foreground and exited on its own: Caddy, Outline, PocketBase, Tandoor Recipes.

Environment, per the sweep's own manifest: GCP e2-standard-2 (2 vCPU, 8 GB), Ubuntu 26.04, Docker installed from get.docker.com. See the how we test page for the wider protocol.

Needs an interactive terminal (11)

The documented command asks a question or uses docker run -it, so an unattended run stops.

AgentArea (ai-agent-platforms) — bootstrap script exits 2 without a terminal after printing the install path; the next step then finds no ./agentarea
LocalAI (ai-chat-interfaces) — the documented `docker run -ti` refuses to start without a terminal
Ollama (ai-chat-interfaces) — the container starts; only the optional `docker exec -it … ollama run` chat step needs a terminal
Paperless-ngx (documents) — the official installer is an interactive questionnaire (URL, port, paths)
Documenso (e-signature) — `openssl req` prompts for certificate fields; adding -subj would make it unattended
ownCloud (file-sync-storage) — the `ocis init` step is `docker run --rm -it` and prints the admin password to the terminal
Beszel (monitoring) — the hub starts; the agent installer asks for the hub's SSH key, which the reader copies from the UI
Glances (monitoring) — the curses terminal UI needs a terminal by design
Zabbix (monitoring) — `mysql -p` prompts for the password the reader chose in the previous step
Ente (photo-backup-gallery) — quickstart.sh ends by asking "Do you want to start Ente? (y/n)"
GoatCounter (web-analytics) — the container starts; only the optional `docker exec -it` site-creation step needs a terminal

Needs a hand-edited file first (8)

The page tells the reader to write or edit a config file between steps; the runner only executes what is written.

OpenCloud (file-sync-storage) — `.env` must be edited with your domains and admin password, and each domain needs a DNS record
Seafile (file-sync-storage) — assumes a root shell (`mkdir /opt/seafile`) and a hand-edited `.env`
Flarum (forum) — assumes a root shell and a PHP host with Composer already installed, as the snippet's first comment says
Chatwoot (helpdesk) — `.env` and the compose file must be edited (secrets, URLs) with nano before start
Psono (password-manager) — settings.yaml has to be written from the printed keys before the migrate step; without it Docker mounts a directory and the container fails
Tandoor Recipes (recipes) — every step exits 0, but PostgreSQL crash-loops until you fill in POSTGRES_PASSWORD and SECRET_KEY in .env, as the snippet says
Caddy (reverse-proxy) — compose starts, but Caddy restart-loops until you write ./conf/Caddyfile, which the snippet leaves to you
Outline (wiki-docs) — compose exits 0, but Outline and its Postgres exit within seconds until you supply real secrets and an auth provider (the snippet's first line says so)

Runs in the foreground (6)

The documented command starts the server and never returns, so an unattended runner waits forever.

OpenHands (ai-coding-agents) — the server starts ("Application startup complete") but the documented command runs in the foreground and never returns; native install, so no container to measure
PocketBase (backend-baas) — `./pocketbase serve` starts the server ("Server started" at 19:13:03), which then exited 0 about 2.5 minutes later for a reason the log does not show. A foreground server is not an unattended pass
GitBucket (code-hosting) — setup steps pass; `java -jar gitbucket.war` starts the server in the foreground (log shows it initializing) and never returns
Frappe CRM (crm) — `docker compose up` without -d: the stack comes up but the command never returns
Zoraxy (reverse-proxy) — the binary starts ("Dynamic Reverse Proxy service started") but `./zoraxy_linux_amd64` runs in the foreground; the snippet's systemd step comes after it
Keycloak (sso-identity) — the dev-mode quickstart starts ("Listening on: http://0.0.0.0:8080") but runs in the foreground by design

Needs a real domain or TLS (1)

The installer validates the hostname or email, or needs ports 80/443 on a public DNS name.

ERPNext (erp) — easy-install.py rejects example.com addresses: "Emails with example.com not acceptable"

Too big for the test box (1)

The install ran out of memory or time on the 8 GB box.

Zammad (helpdesk) — first-boot migrations on 2 vCPU outlast the railsserver healthcheck, so compose exits "unhealthy" and leaves nginx unstarted; about 4 minutes later a second `docker compose up -d` brings the UI up (HTTP 200). Snippet now says so

Our snippet was wrong (1)

A transcription error on our side.

restic (backup) — bunzip2 is not in the stock Ubuntu 26.04 cloud image; snippet now installs bzip2 first. Later steps need your own S3 credentials, which the harness doesn't have

Cause not established (1)

The install failed and the cause was not pinned down in that run; not attributed to upstream or to us.

Swetrix (web-analytics) — the run emitted more than ~512 MB of output and crashed our harness (Node string limit); cause not established — likely a prompt loop in configure.sh, unconfirmed

Failures by category

file-sync-storage3 of 5 swept apps in this category did not pass unattended

monitoring3 of 5 swept apps in this category did not pass unattended

ai-chat-interfaces2 of 5 swept apps in this category did not pass unattended

helpdesk2 of 2 swept apps in this category did not pass unattended

reverse-proxy2 of 4 swept apps in this category did not pass unattended

web-analytics2 of 6 swept apps in this category did not pass unattended

ai-agent-platforms1 of 1 swept apps in this category did not pass unattended

ai-coding-agents1 of 1 swept apps in this category did not pass unattended

backend-baas1 of 4 swept apps in this category did not pass unattended

backup1 of 2 swept apps in this category did not pass unattended

code-hosting1 of 6 swept apps in this category did not pass unattended

crm1 of 8 swept apps in this category did not pass unattended

documents1 of 1 swept apps in this category did not pass unattended

e-signature1 of 2 swept apps in this category did not pass unattended

erp1 of 2 swept apps in this category did not pass unattended

forum1 of 1 swept apps in this category did not pass unattended

password-manager1 of 4 swept apps in this category did not pass unattended

photo-backup-gallery1 of 1 swept apps in this category did not pass unattended

recipes1 of 2 swept apps in this category did not pass unattended

sso-identity1 of 1 swept apps in this category did not pass unattended

wiki-docs1 of 6 swept apps in this category did not pass unattended

Category counts are small (every swept category has fewer than ten apps), so read them as a list of where the failures were, not as a failure rate per category. Categories are the dataset's category field.

Later sweeps (2026-09-29)

Two later sweeps installed further catalog apps. Only the outcome counts are computed here; the per-app cause lives in each app's transcript under docs/guide-validation/ and has not been classified for this report.

2026-09-29 sweep: 50 apps; failed at least once: HeyForm, Zulip; still failing at the end: none
2026-09-29 second sweep: 50 apps; failed at least once: Teable; still failing at the end: none; skipped: flowise (project discontinued: repo archived 2026-08-13, support ended 2026-08-31 (FlowiseAI/Flowise discussion #6727))

Possible upstream defects

While reading transcripts we wrote down 17 places where the upstream project, not our snippet, looked wrong. By kind: compose race (2); broken example (1); docs mismatch (1); log level (1); docs uid (1); compose uid (1); healthy while broken (1); docs gap (1); dead image tag (1); stale v3 volume path (1); docs vs compose (1); compose ordering (1); image never published (1); compose env vars missing (1); docs vs code, no LICENSE (1); shared APP_KEY, no healthcheck (1).

17 of 17 are not filed. They are candidates drawn from our own notes, with the evidence path for each. We check each against the current upstream release before reporting it, and do not list them by project here until that is done.

Not attempted

12 apps were left out of the 2026-09-28 sweep up front because the harness cannot run them. They are not failures:

Appwrite — interactive TTY installer (docker run -it)
Bitwarden — interactive installer needs a bitwarden.com installation id/key
Discourse — interactive ./discourse-setup needs a real domain + SMTP
FreeScout — interactive installer
GitLab CE — 16 GB documented minimum > 8 GB box
Headscale — needs TLS on 443 with a real domain
Jan — known broken make target (2026-07-25)
Mailu — config comes from an interactive web wizard
NetBird — needs a real domain + TLS
OpenPanel — interactive ./setup prompts
Pangolin — interactive installer + wildcard DNS
vLLM — needs NVIDIA GPU

Limits

Our selection, not a sample. We chose the apps, so no failure rate here generalises to self-hosted software.
One machine shape and one OS. An app that failed on Ubuntu 26.04 with current Docker may work elsewhere, and the reverse.
Unattended is the test. Several failures (terminal prompts, hand-edited files) are things a human at a keyboard would get through. They are failures for automation and for a reader who skips a step, not for a reader who follows every one.
Causes are hand-read. The classification is a person's reading of a transcript at the time. It can be wrong, and the notes quote what the transcript showed.
Later sweeps are counts only. Their causes are not classified here.
Point in time. Upstream fixes land; a failure listed here may pass on the next run. Each row's date is in the repository transcripts.

Get the data

A page for each app that passed carries its install-verified date and OS in our open dataset, selfhost-atlas-data (CC BY 4.0, JSON and CSV, DOI 10.5281/zenodo.22983606; the export read for categories was generated 2026-10-13). An app without a stamp has install_verified_date null. The failure causes on this page are published here and in the repository transcripts, not as dataset columns.