How to Deploy AnythingLLM on a VPS

copyapt update && apt upgrade -y
adduser deploy
usermod -aG sudo deploy

copyufw allow OpenSSH
ufw allow 80
ufw allow 443
ufw enable

copycurl -fsSL https://get.docker.com | sh
usermod -aG docker deploy

#Install AnythingLLM

Create a working directory:

copymkdir ~/anythingllm && cd ~/anythingllm

Create a compose file:

copyservices:
  anythingllm:
    image: mintplexlabs/anythingllm:latest
    restart: unless-stopped
    ports:
      - "127.0.0.1:3001:3001"
    volumes:
      - anythingllm_data:/app/server/storage
      - anythingllm_uploads:/app/client/hotdir
    environment:
      - STORAGE_DIR=/app/server/storage
      - JWT_SECRET=$(openssl rand -hex 32)
    cap_add:
      - SYS_ADMIN

volumes:
  anythingllm_data:
  anythingllm_uploads:

The JWT_SECRET is auto-generated on first run. For persistence, generate it once and add it to a .env file:

copyecho "JWT_SECRET=$(openssl rand -hex 32)" > .env

Then reference it in the compose file:

copyenvironment:
  - STORAGE_DIR=/app/server/storage
  - JWT_SECRET=${JWT_SECRET}

Start it:

copydocker compose up -d

AnythingLLM listens on port 3001. The 127.0.0.1 bind keeps it private.

#Connect a model backend

AnythingLLM needs a model backend to generate responses. It supports several options:

Ollama (recommended for self-hosting): If Ollama is on the same server, AnythingLLM can reach it at http://host.docker.internal:11434. If it's on a different server, use the actual URL.

OpenAI API: If you prefer cloud models, you can use your OpenAI API key. This sends data to OpenAI's servers, so it's not fully self-hosted.

Other backends: AnythingLLM also supports Anthropic, Azure OpenAI, LM Studio, and custom endpoints. See the AnythingLLM docs for configuration details.

To configure the backend:

1. Load the AnythingLLM web interface
2. Go to Settings → LLM
3. Select your provider (Ollama, OpenAI, etc.)
4. Enter the connection details
5. Select a model

If you're using Ollama, make sure at least one model is pulled:

copydocker exec -it ollama ollama pull llama3.2

#HTTPS + domain

AnythingLLM needs HTTPS for production use. Point a reverse proxy at 127.0.0.1:3001 and terminate HTTPS on 443.

The simplest path is Automatic HTTPS with Caddy. Point an A record for your hostname (say ai.example.com) at the server's public IP, then have Caddy reverse-proxy that hostname to 127.0.0.1:3001.

If you're using the Caddy container approach, put AnythingLLM and Caddy in the same compose file and proxy to the AnythingLLM service name:

reverse_proxy anythingllm:3001

#First-run setup

Load https://ai.example.com in your browser. On a fresh install, AnythingLLM shows an onboarding wizard.

Create an admin account immediately. The wizard walks you through:

1. Setting an admin email and password
2. Choosing a model backend
3. Configuring vector database settings

Once you're in:

1. Create a workspace — this is a container for documents and conversations. Name it something descriptive (e.g., "Company Docs" or "Project Notes").
2. Upload documents — drag and drop files into the workspace. AnythingLLM indexes them automatically, creating vector embeddings for semantic search.
3. Start chatting — ask questions about your uploaded documents. The AI retrieves relevant chunks and generates answers based on your data.

#Document management

AnythingLLM's strength is document handling. Key features:

Supported formats:

• PDF, DOCX, TXT, CSV, XLSX
• Code files (JS, Python, etc.)
• URLs (scrapes and indexes web pages)

Embedding process: When you upload a document, AnythingLLM:

1. Splits it into chunks (typically 500-1000 tokens each)
2. Generates vector embeddings for each chunk
3. Stores them in the local vector database
4. Makes them available for semantic search during chat

Workspace isolation: Each workspace has its own document collection and chat history. Documents in Workspace A aren't accessible from Workspace B, so you can organize content by project or team.

#Backups

AnythingLLM stores everything in the data volume:

copydocker run --rm -v anythingllm_data:/data -v $(pwd):/backup alpine \
  tar czf /backup/anythingllm-$(date +%F).tar.gz -C /data .

This backs up your documents, embeddings, chat history, and settings. The backup is self-contained — restore it to a fresh AnythingLLM instance and everything comes back.

The uploads volume contains temporary files during processing and can be skipped in backups.

#Upgrades

Pull the newer image and recreate:

copydocker compose pull
docker compose up -d

AnythingLLM runs database migrations automatically. Check the AnythingLLM changelog for breaking changes before a major version bump.

#Troubleshooting

Can't connect to Ollama. Verify Ollama is running (docker compose ps in the Ollama directory) and the URL is correct. If both are on the same server, http://host.docker.internal:11434 should work. Check Docker logs for connection errors.

Document upload fails or hangs. Check disk space — vector embeddings can grow large. Also verify the file isn't corrupted. For large files (>50 MB), try splitting them.

Responses don't match my documents. The model may not be finding the right chunks. Try rephrasing your question, or check that the documents were properly indexed (Workspace → Documents tab shows indexed files).

Out of memory during indexing. Large document collections can spike memory usage. If you're on a small box, index documents in smaller batches rather than uploading everything at once.

"Unauthorized" errors. Your JWT secret may have changed. If you regenerated it, you'll need to log in again with the admin credentials.

#Verification + next steps

You're done when you can: load the web interface over HTTPS, create a workspace, upload documents, and get coherent answers about your data. The chat should reference specific documents and provide relevant excerpts.

From here, explore advanced features like custom prompt templates, API access for integration with other tools, and multi-user workspaces with document permissions. For a simpler chat interface, see Open WebUI. For document-based workflows with automation, combine AnythingLLM with n8n. A 2 vCPU / 4 GB box handles the UI with a remote backend; step up to 8 vCPU / 16 GB for local Ollama. See Best VPS for AI & ML Workloads for the ranked picks.