AnythingLLM Self-Hosted

AnythingLLM Self-Hosted with SQLite

The default in the standard image: one file in the storage directory holding users, workspaces, chats, and settings, with nothing else to run. It suits a single server well, and a backup is a copy of the storage directory. It also means one container per deployment, since SQLite can't be shared between instances.

AnythingLLM Self-Hosted with PostgreSQL

Through AnythingLLM's pg image, which stores all of its data in a PostgreSQL database set with DATABASE_URL and uses pgvector there for document vectors by default. The database needs the pgvector extension installed. Pick it when the team already backs up and operates PostgreSQL, or wants the data in a managed database. The choice is made at install, since moving from the SQLite file later means a data migration.

AnythingLLM Self-Hosted with Ollama

The default and AnythingLLM's most common local runtime: select Ollama as the provider with its base URL, http://host.docker.internal:11434 when both run on one machine or the GPU machine's address otherwise, and pick a pulled model. AnythingLLM reads each model's context length from Ollama and uses up to 16,384 tokens of it, so raise the token limit for long documents when the graphics card has memory to spare, and raise the response timeout from its five-minute default on slow hardware. Ollama can also serve the embedding model in place of the built-in one.

AnythingLLM Self-Hosted with LM Studio

A desktop app with a local server on port 1234 that AnythingLLM connects to through its LM Studio provider, with an optional auth token. It fits a team whose model runs on a workstation or a Mac with plenty of memory on the same network, and it lets someone compare models in a graphical browser before settling on one. A desktop app is the wrong home for a model the whole team depends on all day.

AnythingLLM Self-Hosted with llama.cpp

The lightest runtime: llama.cpp's server exposes an OpenAI-compatible API that AnythingLLM reaches through its Generic OpenAI provider, with the /v1 address, the model name, and the token limit, which falls back to 4,096 when left empty. It runs one model on a CPU or a modest graphics card with very little overhead, and there is no model library or download manager, so you fetch the GGUF file yourself.

AnythingLLM Self-Hosted with vLLM

For a GPU server that answers many people at once: AnythingLLM's Generic OpenAI provider points at vLLM's /v1 address with its API key. Batched serving keeps response times steady when a whole team asks questions at the same time, which single-user runtimes don't. vLLM serves one model per server; vLLM Inference Server is a stack of its own.

Deploy AnythingLLM Self-Hosted on Hetzner

The default. A shared-vCPU server with 2 vCPUs and 4 GB of RAM costs about €19 a month and runs AnythingLLM with room for its built-in embedder and vector store; its x86 CPUs support the AVX2 instructions the vector store needs. That server can run only small models slowly on its CPU, so point AnythingLLM at Ollama on a machine with a graphics card, such as one you own or Hetzner's GPU servers from about €214 a month. Install Docker, mount the storage directory, and set a password before publishing anything.

Deploy AnythingLLM Self-Hosted on Hostinger

The budget entry. The KVM 2 plan has 2 vCPUs, 8 GB of RAM, and 100 GB of NVMe at $8.99 a month on a two-year term, $14.99 at the standard rate, double what AnythingLLM needs, and enough to try a small quantized model on the CPU next to it, slowly. The cheaper KVM 1 has a single vCPU, below the 2-core minimum. Root access is full and the panel adds weekly backups; TLS and the password are your job.

Deploy AnythingLLM Self-Hosted on DigitalOcean

AnythingLLM maintains a Terraform template that creates a Droplet running the container, served over plain HTTP on port 3001, so a password and a proxy for HTTPS still come after it. The template's default is the 2 GB Droplet at $18 a month, AnythingLLM's floor; the 4 GB Basic Droplet with 2 vCPUs at $24 covers the app with headroom. DigitalOcean also rents GPU Droplets, which can host the model runtime near the app when a team has no hardware of its own.

Deploy AnythingLLM Self-Hosted on Railway

A container platform with a one-click template linked from AnythingLLM's README, using the Railway build of the image. TLS ends at the platform, so the reverse proxy or PaaS choice doesn't apply, but the instance still needs a password before it is shared. Attach a volume at the storage directory, or the database and documents go with each redeploy. Memory is billed at about $10 per GB a month on top of the $5 Hobby plan, so the app lands around $25. There is no GPU here, so the model runs elsewhere or is a hosted one.

Deploy AnythingLLM Self-Hosted on Render

A container platform with a Deploy to Render button in AnythingLLM's README, built from the render branch and the Render build of the image. AnythingLLM's 2 GB floor means the Standard instance, $25 a month, plus a persistent disk for the storage directory, which also keeps the service to a single instance. TLS ends at the platform, so the reverse proxy or PaaS choice doesn't apply. Render has no GPUs, so pair it with a remote runtime or a hosted model.

Deploy AnythingLLM Self-Hosted on Amazon Web Services

AnythingLLM's own CloudFormation template creates one EC2 instance with a 10 GiB volume and serves the app over plain HTTP on port 3001, so a password, a domain, and HTTPS are still to add. Its default t3.small has 2 GB, AnythingLLM's floor; a t3.medium with 2 vCPUs and 4 GB costs about $30 a month on demand. Bedrock is one of AnythingLLM's model providers, so a team on AWS can keep model calls in the same account, and GPU instances can host a runtime for local models.

Deploy AnythingLLM Self-Hosted on Google Cloud Platform

AnythingLLM's GCP template creates a Compute Engine VM, but its defaults need changing: an n1-standard-1 has a single vCPU, below the 2-core minimum, it uses Ubuntu 20.04, and it opens no firewall rule for port 3001. Pick a machine with 2 vCPUs and a current image instead, about $25 to $40 a month left on, and allow the port only from your own addresses. Vertex AI is one of AnythingLLM's model providers, so a team on Google Cloud can keep hosted models in the same project.

AnythingLLM Self-Hosted with Traefik

The family default: it discovers the AnythingLLM container from Docker labels and renews certificates itself, with one router for port 3001. Raise the response timeouts, because streamed answers from a slow local model can run for minutes and a proxy that cuts them breaks the chat, and raise the upload size limit so people can add large documents. Set the password or multi-user mode before the router goes live.

AnythingLLM Self-Hosted with Caddy

Automatic HTTPS from one site block that forwards to port 3001, with streamed answers passed through as they arrive and generous default timeouts for slow models. The simplest route from the official cloud templates' plain HTTP to a proper domain with TLS. Turn on password protection or multi-user mode before the address is shared.

AnythingLLM Self-Hosted with NGINX

The proxy many servers already run. Turn proxy_buffering off so streamed answers aren't held back, raise proxy_read_timeout past the 60-second default for slow local models, and raise client_max_body_size, whose 1 MB default rejects most document uploads. Forward the Upgrade and Connection headers for the app's websocket connections.

AnythingLLM Self-Hosted with Coolify

A free, self-hosted deploy platform with an AnythingLLM template in its service library. The template generates the password, the JWT secret, and the signing key and salt, turns telemetry off, and keeps the built-in vector store, so the instance is protected from its first start. It runs the latest image, which changes with almost every commit, so pin a versioned tag. Coolify's own proxy handles the domain and TLS.

AnythingLLM Self-Hosted with Dokploy

Free and self-hosted, with an AnythingLLM template that serves the container on your domain with HTTPS through its Traefik routing and keeps the storage directory on a named volume. Unlike Coolify's, it sets no password or secrets and runs the latest image, so pin a versioned tag and set a password or multi-user mode at the first visit, before sharing the address. Domains and TLS come with the platform, so no separate reverse proxy is added next to it.