how to host Ollama for VPS safely

Wetin you dey build

One single open-weight language model wey go run for server wey you own. You fit use HTTP API or you fit use chat page for your browser to answer questions. Ollama na the part wey go download the model, load am for memory, and serve requests for http://127.0.0.1:11434. You fit install am with just one command. All the hard work dey elsewhere: how to pick model wey your VPS RAM fit carry, and how to make sure say you no accidentally publish unauthenticated inference server for the whole internet.

Two honest warnings before we start. CPU-only VPS go run small models slow, and the API no get built-in authentication at all. We go explain both of dem well-well below, because na there many people dey get wahala.

The sizing reality check, in plain numbers

A model memory footprint na roughly its file size, plus about one gigabyte for runtime overhead, plus some more for the context window. Ollama default models na 4-bit quantized (dem label am as Q4), which means e go take about half a gigabyte of RAM for every billion parameters. So the math simple, and na wetin e do na e go decide everything.

3B model like llama3.2:3b na ~2 GB download and e go need around 4 GB of free RAM to run. 7B or 8B model like mistral:7b or llama3.1:8b na ~5 GB for disk and e go need about 8 GB of RAM, but 16 GB go make am dey comfortable. 13B or 14B model go need roughly 16 GB. Anything for 30B-to-70B range need big-RAM box or, if we dey talk true, GPU — for CPU VPS, e either no go fit or e go answer so slow wey e no go useful.

Now for speed, because na this part people dey underestimate. CPU inference depend on memory bandwidth, no be clock speed, and shared vCPU VPS get small bandwidth. Expect single-digit to low-double-digit tokens per second: 7-8B Q4 model fit get 4 to 10 tokens per second, 3B model na 10 to 25. GPU faster by roughly one order of magnitude. These figures na rough estimate by design — the correct way na to measure your own box, wey the run step below go show you how to do. Trust your eval rate, no be any number for article, even this one.

The practical conclusion: small quantized models on CPU dey useful for drafting, summarizing, and classification if you fit accept the speed. For anything bigger or faster, budget for GPU instance.

To weigh one specific model against one specific box, estimate its memory footprint here:

Install Ollama

Two clean ways dey exist. Official script na simplest one if you dey use bare VPS:

curl -fsSL https://ollama.com/install.sh | sh

Dis one go create system user wey dem call ollama, install binary for /usr/local/bin/ollama, and register systemd service wey dem call ollama.service. Dis service go start automatically when computer boot and e go bind 127.0.0.1:11434. Check if e dey work:

systemctl status ollama
ollama --version

If you don dey use Docker already, use container version:

docker run -d --name ollama \
  -p 127.0.0.1:11434:11434 \
  -v ollama:/root/.ollama \
  --restart always \
  ollama/ollama

Watch dis 127.0.0.1: prefix for di port mapping. Dis one dey bind di port to localhost only. If you write -p 11434:11434, e go publish di port for every interface, and na dis mistake di security section warn about. Pick one install method; no run di script and di container together, or two processes go fight for di port.

Pull and run your first model

ollama pull llama3.2:3b
ollama run llama3.2:3b

pull go download the model layers to disk (about 2 GB for this one). run go load dem into memory and carry you go >>> prompt. Type question. The first token fit take several seconds as the weights dey load from disk enter RAM, then the answer go start to stream. Type /bye if you wan leave the chat; Ollama go still dey run for background.

See wetin load and how e fit:

ollama ps

The PROCESSOR column na him dey tell truth. 100% CPU mean say no GPU dey work, and na why e dey slow. Measure real speed with the verbose flag:

ollama run --verbose llama3.2:3b "Write two sentences about Linux."

The eval rate line wey print for end na your tokens per second for this hardware. Na that number you go use plan.

Where models live, and how much disk to buy

Script install am and e dey run as service, models dey inside ollama user home directory:

sudo du -sh /usr/share/ollama/.ollama/models

If you run am interactively as your own user, dem dey sit for ~/.ollama/models. Inside container, dem dey inside ollama named volume. Dis thing important because quantized weights dey grow fast: 3B na ~2 GB, 7-8B na ~5 GB, 14B na ~9 GB. If you pull four models to compare dem, you don use 20 GB without even notice. Calculate disk size based on models wey you wan keep, and use ollama rm <model> delete de rest.

Run am as service wey you control

The install script don already register ollama.service, so e go restart itself when computer boot without extra work. One setting wey worth change na how long model go stay for memory, and for some setups, the bind address — both of dem dey inside systemd drop-in so say Ollama upgrade no go overwrite dem:

sudo systemctl edit ollama.service

Add this one under the [Service] header wey editor show you:

[Service]
Environment="OLLAMA_KEEP_ALIVE=30m"

OLLAMA_KEEP_ALIVE na how long model go stay for memory after last request (default na 5 minutes). Increase am if you dey query one box all day so that e no go reload the weights every time; set am to 0 for small box wey get tight RAM so e fit free RAM as soon as request finish. systemctl edit go reload the unit files for you, so restart to make the change work:

sudo systemctl restart ollama

The security point wey matter pass

By default, Ollama bind 127.0.0.1:11434, so only processes wey dey inside the VPS itself fit reach am. That default correct. No change am.

The API no get authentication. None. No API key, no login, no rate limit, and no allow-list. Anybody wey fit reach port 11434 fit run any model wey you don pull, pull new ones, delete dem, and pin your CPU or GPU at full load indefinitely. Scanners like Shodan dey index open Ollama instances by the thousand, and if dem find one wey open, dem go abuse am within hours.

So, dis na the single mistake wey you must avoid: no set OLLAMA_HOST=0.0.0.0 and no open 11434 for your firewall. Dat one go publish unauthenticated inference server to the whole internet. No configuration fit make raw 11434-on-0.0.0.0 safe, because Ollama no get anything to configure — authentication simply no dey exist.

Dem get three safe ways to reach the model from somewhere else besides the box:

• Keep am local. If na only another program for the same VPS dey call am — like cron script, bot, or MCP server wey bridge your tools to the model — leave the bind for 127.0.0.1 and make dat program call http://127.0.0.1:11434. Nothing go dey exposed and nothing else dey needed.
• Reach am through private tunnel. Put the VPS for WireGuard VPN wey you host yourself, set OLLAMA_HOST to the tunnel address (for example 10.8.0.1, no be 0.0.0.0), and only VPN peers fit connect. Public internet still no go see anything for 11434.
• Put authenticating reverse proxy for front. Terminate TLS and demand password or token for nginx, Traefik, or Caddy, then proxy am to 127.0.0.1:11434. Ollama go keep its localhost bind; na the proxy be the only thing wey dey listen for public port. Dis one na same way as when you put Let's Encrypt certificate for nginx for front of any local service.

The reverse-proxy option na exactly wetin the chat UI go give you next, with real login attached.

Add a chat UI with Open WebUI, behind TLS

Open WebUI na self-hosted chat interface. Run am inside Docker and point am to local Ollama:

docker run -d \
  --name open-webui \
  --network=host \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  -v open-webui:/app/backend/data \
  --restart always \
  ghcr.io/open-webui/open-webui:main

--network=host flag na important detail for Linux VPS. E dey put container inside host network namespace, so 127.0.0.1 inside container na host loopback, and container go reach Ollama for 127.0.0.1:11434 even if Ollama no dey listen for any other interface. That bridge-network method wey dem dey talk elsewhere — --add-host=host.docker.internal:host-gateway with OLLAMA_BASE_URL=http://host.docker.internal:11434 — no go work for here: that name dey point to Docker bridge gateway, and service wey bind to 127.0.0.1 for host no dey reachable across the bridge, so Open WebUI go just sit down dey report say e no fit connect to Ollama.

The trade-off for host networking na say Open WebUI go dey listen for host port 8080 for every interface; any -p mapping go no work, and Docker go print warning say so. So close 8080 for both host and provider firewall, and make TLS reverse proxy be the only public door. For the very first visit, Open WebUI go ask you to create admin account — that account na your authentication layer, so pick strong password.

To open the chat from your laptop over HTTPS, put TLS reverse proxy for front of 127.0.0.1:8080. If you don already route many Docker apps for the box, Traefik with automatic TLS across many apps na the cleanest way: one label block go issue certificate and route chat.example.com to Open WebUI. The rule from security section still stand — proxy go carry public port and login, while Ollama go stay for localhost and Open WebUI own 8080 go stay firewalled.

Use the OpenAI-compatible endpoint from your code

Ollama dey speak one subset of the OpenAI chat API for /v1. Because of this, most OpenAI client libraries go work if you change two things: the base URL and one throwaway key.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="ollama")

resp = client.chat.completions.create(
    model="llama3.2:3b",
    messages=[{"role": "user", "content": "Name three Linux distributions."}],
)
print(resp.choices[0].message.content)

The client library need the api_key but Ollama no dey use am, so any string go work. model must be one name wey you don already pull; if the name no dey, e go return model "x" not found, try pulling it first. You fit use plain curl call for the same thing:

curl http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Hello"}]}'

Na so you go wire the model into agent and editor tooling. If you dey develop for the box already, local model fit back scripts and plugins alongside Claude Code wey dey run for VPS inside tmux, so you fit keep cheap, private drafting work off paid API while heavy reasoning stay with hosted model.

Failure modes, with the exact strings you will see

The process is "Killed" mid-generation. You start one big model and terminal go print Killed, or server log go show llama runner process has terminated: signal: killed. Linux OOM killer stop am because the model need more RAM than the box get. Confirm the cause with sudo dmesg | grep -i oom, where you go see one line like Out of memory: Killed process ... (ollama). The fix na to use smaller model or model wey get heavy quantization — llama3.2:3b instead of 13B — or add swap so that load wey just overflow physical RAM fit survive slowly instead of die. Swap go turn instant crash to slow answer; e no go make 70B model practical for 4 GB.

"Error: model requires more system memory". Ollama refuse to start the model and print Error: model requires more system memory (X GiB) than is available (Y GiB). This one na the polite version of the crash wey I talk before: Ollama do the arithmetic and stop itself instead of letting the OOM killer stop am. E even give you the two numbers. Pick one model wey requirement small pass your free RAM (check am with free -h), reduce the context length, or move to one bigger VPS. No flag fit make the model fit — the memory na real thing.

The first token takes forever, then it is fine. Cold model go print nothing for five to thirty seconds, then e go stream normally. That pause na because the weights dey load from disk into RAM for the first time, and slow storage go make am worse. Once e load, the model go stay resident for the length of OLLAMA_KEEP_ALIVE, so the second prompt go answer instantly. Increase that value if the gaps dey annoy you, and use ollama ps to see whether one model dey loaded currently.

Everything is simply slow. Ten tokens per second or less, with no error at all. That one na CPU inference dey do exactly wetin CPU inference dey do. ollama ps show 100% CPU, meaning say no GPU dey. This one no be bug and no setting fit fix am, because the limit na memory bandwidth, no be misconfiguration. Use smaller model, accept the speed, or move to one GPU instance — and measure your real rate with --verbose before you decide say anything broken.

Connection refused from another machine. From your laptop, you go get curl: (7) Failed to connect to <ip> port 11434: Connection refused. This one dey work as designed: Ollama bind only to localhost. No "fix" am by binding 0.0.0.0, because na that one be the exposure mistake wey I talk before. Reach the model over the VPN or through the authenticating proxy instead.

You exposed 11434 to the internet. If you set OLLAMA_HOST=0.0.0.0, open the firewall, and now you see model pulls wey you no start or CPU dey pin at 100% because of unknown clients, dem don find you and dem don use you. This one na the main mistake, no be edge case. Rebind to 127.0.0.1 or the VPN address, close 11434 for firewall, and put authentication for front. Assume say anything wey reach that address while e open, strangers don query am.

Backups and upgrades

No big thing to lose for here. You fit re-download all the models, so na only Open WebUI data volume — accounts, chat history, and settings — and any systemd drop-in wey you write, na dem worth to back up. Use throwaway container back up the volume:

docker run --rm -v open-webui:/data -v "$PWD":/backup alpine \
  tar czf /backup/open-webui.tgz -C /data .

To upgrade Ollama, just run the install script again; to upgrade Open WebUI, use docker pull ghcr.io/open-webui/open-webui:main then recreate the container. No pin anything for long-term: model quality and runtime dey change fast, so read the release notes and run benchmark again for your own box instead of to trust last quarter numbers.

FAQ

I fit really run LLM for CPU-only VPS?

Yes, but e get limit. Small quantized models wey dey 3B to 8B range dey run for CPU and dem dey useful for draft, summary, and classification — but e slow, e dey produce single-digit to low-double-digit tokens per second for shared vCPU. Anything from 13B upward go slow well-well or e no go fit enter RAM at all. If you want speed or big models, you need GPU instance.

How much RAM each model need?

Rough rule for default 4-bit quantized models: about 0.5 GB of RAM per billion parameters for weights, plus roughly 1 GB for overhead and small extra for context. So 3B model need about 4 GB free, 7-8B model need about 8 GB, and 14B model need about 16 GB. Check your headroom with free -h and make sure you leave space for operating system and other things for the box.

Ollama API get authentication?

No. Ollama no get built-in authentication, API key, or rate limit — anybody wey fit reach port 11434 get full control. Na why e bind 127.0.0.1 by default and why you must never expose 11434 on 0.0.0.0 to the internet. Use local access, private VPN, or reverse proxy wey get login.

How I fit add web chat interface?

Run Open WebUI for Docker with --network=host so e fit share host loopback and reach native Ollama for http://127.0.0.1:11434, then put TLS reverse proxy for front of its port 8080 for you to access from your laptop. Keep 8080 closed for firewall so that proxy na the only public door. Open WebUI admin account na him dey give login, and you go set password when you first launch am.

How I fit call am from my own application?

Use the OpenAI-compatible endpoint for http://127.0.0.1:11434/v1. Point any OpenAI SDK to that base URL, pass any string as the API key because e no dey use am, and set model to the name wey you don pull. Existing OpenAI code dey run normally, only the base URL and key go change.