How to host Ollama for VPS LLM safely

Wetin you dey build

Na one open-weight language model wey dey run for server wey you own. You fit answer am through HTTP API, and if you want, use chat page for your browser. Ollama na the part wey download the model, load am into memory, and serve requests for http://127.0.0.1:11434. Installation na one command. The hard parts dey elsewhere: choosing model wey your VPS fit hold for RAM, and making sure you no accidentally publish unauthenticated inference server to the whole internet.

Make we talk two honest warnings first. CPU-only VPS dey run small models slowly, and API no get built-in authentication at all. We cover both well-well below, because na these two places people dey get serious problem.

The sizing reality check, for plain numbers

Model memory footprint near-be its file size, plus about one gigabyte runtime overhead, plus extra memory for the context window. Ollama default models dey use 4-bit quantization (dem label am Q4), and this dey use about half gigabyte RAM for every billion parameters. So the calculation simple, and e dey decide everything.

A 3B model like llama3.2:3b na ~2 GB download, and e need around 4 GB free RAM to run. A 7B or 8B model like mistral:7b or llama3.1:8b na ~5 GB for disk, and e need about 8 GB RAM, or 16 GB to run comfortably. A 13B or 14B model need roughly 16 GB. Anything for the 30B-to-70B range need machine wey get plenty RAM or, realistically, GPU. For CPU VPS, e either no go fit enter memory, or e go answer so slowly say e no useful.

Now make we talk speed, because na this part people dey underestimate. CPU inference depend on memory bandwidth, no be clock speed, and shared vCPU VPS get modest bandwidth. Expect single-digit to low-double-digit tokens per second: 7-8B Q4 model fit manage 4 to 10 tokens per second, while 3B model fit do 10 to 25. GPU dey roughly one order of magnitude faster. These figures na rough figures on purpose. The correct thing na to measure your own box, and the run step below show how to do am. Trust your eval rate, no be number for any article, including this one.

The practical conclusion be say small quantized models for CPU dey genuinely useful for drafting, summarizing, and classification if you fit accept the speed. For anything bigger or faster, budget for GPU instance.

To compare one model with one specific box, estimate the model memory footprint here:

Install Ollama

Ways clean dey two. Official script na the simplest option for bare VPS:

curl -fsSL https://ollama.com/install.sh | sh

Dis go create system user wey dem name ollama, install binary for /usr/local/bin/ollama, and register systemd service wey dem call ollama.service. Service go start when system boot and e go bind to 127.0.0.1:11434. Confirm say e dey up:

systemctl status ollama
ollama --version

If Docker dey already run for your server, use container instead:

docker run -d --name ollama \
  -p 127.0.0.1:11434:11434 \
  -v ollama:/root/.ollama \
  --restart always \
  ollama/ollama

Notice the 127.0.0.1: prefix for the port mapping. Dis one bind the port to localhost only. If you write -p 11434:11434 instead, e go publish the port for every interface. Na this mistake the security section dey warn about. Choose one install method. No run the script and container together, because two processes go dey fight for the port.

Pull and run your first model

ollama pull llama3.2:3b
ollama run llama3.2:3b

pull go download the model layers enter disk (about 2 GB for this one). run load dem enter memory and put you for a >>> prompt. Type one question. The first token fit take several seconds while the weights dey load from disk enter RAM, then the answer go dey stream. Type /bye to comot from the chat; Ollama go continue dey run for background.

See wetin don load and how e take fit:

ollama ps

The PROCESSOR column na the correct one. 100% CPU mean say GPU no dey involved, and na this cause the slowness. Measure the real speed with the verbose flag:

ollama run --verbose llama3.2:3b "Write two sentences about Linux."

The eval rate line wey print for the end na your tokens per second for this hardware. Na this number you suppose plan with.

Model dem dey, and how much disk space you suppose buy

Script install dem and service dey run dem, models dey inside ollama user home:

sudo du -sh /usr/share/ollama/.ollama/models

If you run dem interactively as your own user, dem dey inside ~/.ollama/models. For container, dem dey inside ollama named volume. This matter because quantized weights fit use disk space quickly: 3B na ~2 GB, 7-8B na ~5 GB, and 14B na ~9 GB. If you pull four models to compare dem, you don use 20 GB before you notice. Plan disk size based on the models wey you intend keep, and delete the remaining ones with ollama rm <model>. If the same VPS dey run another thing wey dey need plenty space, like PhotoPrism or Immich wey dey hold photo library, first subtract that space from the available free space. Treat the space wey remain as your real model budget.

Run am as service wey you control

The install script don already register ollama.service, so e go restart when system boot without extra work. The setting wey worth changing na how long model go stay resident, and, for some setups, the bind address. Put both for systemd drop-in so Ollama upgrade no go overwrite dem:

sudo systemctl edit ollama.service

Add this under the [Service] header wey the editor show you:

[Service]
Environment="OLLAMA_KEEP_ALIVE=30m"

OLLAMA_KEEP_ALIVE na how long model go stay for memory after the last request (default 5 minutes). Increase am for machine wey you dey query all day, so e no need reload the weights every time. Set am to 0 for machine wey RAM tight, so e go free RAM immediately request finish. systemctl edit go reload the unit files for you, so restart the service to apply the change:

sudo systemctl restart ollama

The security point wey matter pass

By default Ollama dey bind to 127.0.0.1:11434, so na only processes wey dey on the VPS itself fit reach am. That default correct. Keep am.

The API no get authentication. None at all. E no get API key, login, rate limit, or allow-list. Anybody wey fit reach port 11434 fit run any model wey you don pull, pull new ones, delete dem, and pin your CPU or GPU for full load without stopping. Scanners like Shodan dey index open Ollama instances by thousands, and dem dey find and abuse exposed one within hours.

So na this one mistake you must never make: no set OLLAMA_HOST=0.0.0.0 and open 11434 for your firewall. That one publish unauthenticated inference server to the whole internet. No configuration fit make raw 11434-on-0.0.0.0 safe, because Ollama no get anything wey you fit configure for this matter; the authentication simply no dey exist.

There are three safe ways to reach the model from somewhere wey no be the box:

• Keep am local. If na only another program for the same VPS, a cron script, a bot, or an MCP server wey dey connect your tools to the model dey call am, leave the bind for 127.0.0.1 and make that program call http://127.0.0.1:11434. Nothing dey exposed, and you no need anything else.
• Reach am through private tunnel. Put the VPS for a WireGuard VPN wey you host by yourself, set OLLAMA_HOST to the tunnel address (for example 10.8.0.1, not 0.0.0.0), and na only VPN peers fit connect. The public internet still no go see anything for 11434.
• Put reverse proxy wey get authentication in front. Terminate TLS and require password or token for nginx, Traefik, or Caddy, then proxy to 127.0.0.1:11434. Ollama go keep its localhost bind; na only the proxy go listen on the public port. This na the same arrangement as putting a Let's Encrypt certificate for nginx in front of any local service.

The reverse-proxy option na exactly wetin the chat UI go give you next, with real login attached.

Add chat UI with Open WebUI, behind TLS

Open WebUI na self-hosted chat interface. Run am for Docker and point am to the local Ollama:

docker run -d \
  --name open-webui \
  --network=host \
  -e OLLAMA_BASE_URL=http://127.0.0.1:11434 \
  -v open-webui:/app/backend/data \
  --restart always \
  ghcr.io/open-webui/open-webui:main

The --network=host flag na the important detail for Linux VPS. E dey put the container for the host network namespace, so 127.0.0.1 inside the container na the host own loopback and the container fit reach Ollama for 127.0.0.1:11434 without Ollama listening on any other interface. The bridge-network recipe wey you go see elsewhere, --add-host=host.docker.internal:host-gateway with OLLAMA_BASE_URL=http://host.docker.internal:11434, no work here: that name resolve to Docker bridge gateway, and service wey bind to 127.0.0.1 for the host no dey reachable across the bridge. So Open WebUI just dey report say e no fit connect to Ollama.

The trade-off for host networking be say Open WebUI now dey listen on host port 8080 for every interface. Any -p mapping go discard, and Docker go print warning to explain am. So close 8080 for both the host firewall and provider firewall, then make TLS reverse proxy be the only public door. For the very first visit, Open WebUI go ask you to create admin account. That account na your authentication layer, so choose strong password.

To open the chat from your laptop over HTTPS, put TLS reverse proxy in front of 127.0.0.1:8080. If you already dey route several Docker apps for the box, Traefik with automatic TLS across many apps na the cleanest fit: one label block go issue the certificate and route chat.example.com to Open WebUI. The rule from the security section still apply. The proxy own the public port and the login, while Ollama remain on localhost and Open WebUI own 8080 remain firewalled.

Use OpenAI-compatible endpoint from your code

Ollama dey speak part of OpenAI chat API for /v1, so most OpenAI client libraries go work after you change two things: base URL and one throwaway key.

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="ollama")

resp = client.chat.completions.create(
    model="llama3.2:3b",
    messages=[{"role": "user", "content": "Name three Linux distributions."}],
)
print(resp.choices[0].message.content)

Client library dey require api_key, but Ollama dey ignore am, so any string go work. model must be name wey you don pull before; unknown name go return model "x" not found, try pulling it first. Plain curl call na the same idea:

curl http://127.0.0.1:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"Hello"}]}'

Na so you go connect the model to agent and editor tooling too. If you already dey develop for the box, local model fit support scripts and plugins together with Claude Code wey dey run for the VPS inside tmux, so cheap, private drafting work no go use paid API, while hosted model go handle the heavy reasoning.

Failure modes, with the exact strings you will see

The process is "Killed" mid-generation. You start a large model and the terminal prints Killed, or the server log shows llama runner process has terminated: signal: killed. The Linux OOM killer stop am because the model need more RAM than the box get. Confirm the cause with sudo dmesg | grep -i oom, where you go see a line like Out of memory: Killed process ... (ollama). The fix na smaller or more heavily quantized model, llama3.2:3b instead of a 13B, or add swap so a load wey just pass physical RAM fit survive slowly instead of dying. Swap turn instant crash to slow answer; e no make 70B model practical for 4 GB.

"Error: model requires more system memory". Ollama refuse to start the model and print Error: model requires more system memory (X GiB) than is available (Y GiB). Na the polite version of the crash above: Ollama do the arithmetic and stop instead of letting the OOM killer do am. E even give you the two numbers. Choose model wey requirement dey below your free RAM (check with free -h), reduce the context length, or move to bigger VPS. No flag fit make the model fit; the memory na real.

The first token takes forever, then it is fine. Cold model print nothing for five to thirty seconds, then e stream normally. That pause na the weights loading from disk into RAM for the first time, and slow storage make am worse. Once e load, the model stay resident for the length of OLLAMA_KEEP_ALIVE, so the second prompt answer instantly. Increase that value if the gaps dey annoy you, and use ollama ps to see whether model dey currently loaded.

Everything is simply slow. Ten tokens per second or fewer, with no error at all. Na CPU inference dey do exactly wetin CPU inference dey do. ollama ps show 100% CPU, meaning say GPU no dey available. This no be bug and no setting fit fix am, because the limit na memory bandwidth, not wrong configuration. Use smaller model, accept the speed, or move to GPU instance, and measure your real rate with --verbose before you decide say anything spoil.

Connection refused from another machine. From your laptop you get curl: (7) Failed to connect to <ip> port 11434: Connection refused. This dey work as designed: Ollama bind localhost only. No "fix" am by binding 0.0.0.0, because na exactly the exposure mistake we talk about above. Reach the model through VPN or authenticating proxy instead.

You exposed 11434 to the internet. If you set OLLAMA_HOST=0.0.0.0, open the firewall, and now see model pulls wey you no start or CPU pinned at 100% by unknown clients, dem don find and use your server. This na the main mistake, not edge case. Bind am again to 127.0.0.1 or the VPN address, close 11434 for the firewall, and put authentication in front. Assume say strangers query anything wey dey reachable for that address while e remain open.

Backups and upgrades

State wey fit lost no plenty. You fit download the models again, so na only Open WebUI data volume, accounts, chat history, settings, and any systemd drop-in wey you write dey worth backing up. Back up the volume with one throwaway container:

docker run --rm -v open-webui:/data -v "$PWD":/backup alpine \
  tar czf /backup/open-webui.tgz -C /data .

Upgrade Ollama by running the install script again; upgrade Open WebUI with docker pull ghcr.io/open-webui/open-webui:main, then recreate the container. No pin anything for long term: model quality and runtime dey change fast, so read the release notes and run benchmark again for your own box instead of trusting last quarter's numbers.

FAQ

I fit really run LLM for VPS wey na CPU-only?

Yes, but e get limit. Small quantized models for 3B to 8B range fit run for CPU and dem useful for drafting, summarizing, and classification, but dem dey slow, around single-digit to low-double-digit tokens per second for shared vCPU. Anything from 13B go up go slow well well or e no go fit enter RAM at all. If you need real speed or bigger models, you need GPU instance.

How much RAM each model need?

For the default 4-bit quantized models, rough rule be about 0.5 GB RAM for every billion parameters for the weights, plus around 1 GB overhead and small extra space for context. So, 3B model need about 4 GB free, 7-8B model need about 8 GB, and 14B model need about 16 GB. Check your available headroom with free -h and leave space for the operating system and anything else wey dey run for the box.

Ollama API get authentication?

No. Ollama no get built-in authentication, API key, or rate limit. Anybody wey fit reach port 11434 get full control of am. Na exactly why e bind to 127.0.0.1 by default, and why you must never expose 11434 on 0.0.0.0 to the internet. Reach am locally, through private VPN, or through reverse proxy wey add login.

How I fit add web chat interface?

Run Open WebUI for Docker with --network=host so e share the host loopback and reach native Ollama for http://127.0.0.1:11434. Then put TLS reverse proxy in front of port 8080 so you fit access am from your laptop. Keep 8080 closed for the firewall so the proxy be the only public door. Open WebUI own admin account provide the login, and you go set the password when e launch for the first time.

How I fit call am from my own application?

Use the OpenAI-compatible endpoint for http://127.0.0.1:11434/v1. Point any OpenAI SDK to that base URL, pass any string as the API key because e dey ignored, and set model to a name wey you don pull. Existing OpenAI code usually go run without changes, except for the base URL and key.