How to Run Ollama Rootless with Podman for VPS

Run Ollama for rootless Podman on VPS

To run Ollama for rootless Podman on server, five things must dey correct wey desktop guide fit skip. One dedicated unprivileged user go own the container. You must enable lingering for that user, so container go continue to run after you log out. Quadlet file go hand the container over to systemd, so e go come back after reboot. The model directory must get SELinux label for distributions wey enforce am. The API go listen only on loopback, and you go reach am through SSH (secure shell) tunnel.

Ollama na server for large language models (LLM). E dey store model weights for disk, load dem into memory, and answer HTTP requests for port 11434. E no get login, API key, or user accounts, so na network be the only access control wey you get. Podman dey run containers without daemon and without root, so anything wey escape from the container go first run as ordinary unprivileged user. If you want see the runtime comparison first, read how Podman and Docker dey differ for VPS. If you prefer skip containers completely, how to install Ollama directly for VPS na shorter way.

SSD Nodes dey provision Fedora among its images, and Fedora dey ship both Podman and SELinux (security-enhanced Linux) by default. Every command below go run for any distribution wey get Podman 5 or newer.

Why laptop version need changes for server

Fedora Magazine publish a clear walkthrough of this stack on 5 August 2026: Running Ollama Locally with Podman on Fedora Linux, wey Yazan Monshed write. E good as first-hour introduction to the tools. But e target laptop, and four of the choices behave differently for machine wey get public IP address.

• E start the container with plain podman run -d. Container wey person start by hand no go come back after reboot, because nobody ever configure am to start.
• E use moving tag ollama/ollama. For laptop, you go notice the day wey the behaviour change. For server, the first sign fit be script wey stop working overnight.
• E publish with -p 11434:11434, wey bind every interface. Behind home router, internet no fit reach am. For VPS, e become public inference API wey no get password.
• E run as your own login user. For server, the account wey own the container suppose own nothing else, so if break-out happen, e go land for empty home directory.

None of these things wrong for the machine wey the guide write for. Each one na decision wey you need review when the box dey reachable from everywhere and nobody dey sit in front of am.

Create the unprivileged user, and check subuid

Create the unprivileged user, then check subuid.

Rootless Podman dey map the container internal user IDs (UID) onto one block of unused IDs for the host. Dem declare this block for /etc/subuid and /etc/subgid. Without this block, rootless containers no fit start at all.

sudo dnf install -y podman        # or: sudo apt install -y podman
sudo useradd --create-home --shell /bin/bash --comment "Ollama container owner" ollama
sudo passwd --lock ollama
grep ollama /etc/subuid /etc/subgid

grep suppose print two lines, one from each file. Each line go show one range of 65536 IDs:

/etc/subuid:ollama:100000:65536
/etc/subgid:ollama:100000:65536

Your starting number go different, and that one no be problem. If grep no print anything, useradd no allocate any range. The first podman command wey you run as that user go fail like this:

Error: cannot find UID/GID for user ollama: no subuid ranges found for user "ollama" in /etc/subuid

Assign one range wey no other user dey use, then tell Podman say the old mapping don stale:

sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 ollama
sudo -iu ollama podman system migrate

Locking the password mean say nobody fit log in directly as ollama. You go reach the account from your admin user with sudo -iu ollama.

Enable lingering make service survive logout

A user wey systemd instance normally start when dem login, e dey stop when dem logout, and /run/user/<uid> dey remove together with am. Every rootless container wey that user own go die for the same time. Lingering keep the user instance running even when no session attach to am.

sudo loginctl enable-linger ollama
loginctl show-user ollama --property=Linger

That one suppose print Linger=yes. Enable am before you create the unit, because the directory wey the unit need, /run/user/<uid>, only dey exist after lingering don turn on.

One more step dey wey nobody dey expect. sudo -iu ollama give you shell but e no give you session bus, so systemctl --user fail immediately:

Failed to connect to bus: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not defined

systemd dey look for the user bus for $XDG_RUNTIME_DIR/bus, and sudo -i no set that variable. Set am by hand for every admin shell where you manage this service:

sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user status

Wia model blobs dey land, and how much disk you suppose plan

Ollama dey write weights inside /root/.ollama/models for the container. Bind one directory from the user's home go that path, so the files go land for place wey you fit measure: /home/ollama/ollama-data/models. Blobs dey enter models/blobs as content-addressed files, while models/manifests dey hold the small index wey name dem. If you use named volume instead, like Fedora Magazine post do, the same tree dey under /home/ollama/.local/share/containers/storage/volumes/<volume>/_data.

Size the disk before you pull anything. Published download sizes give you the minimum.

[
  {
    "label": "gemma3:4b",
    "download_gb": 3.3
  },
  {
    "label": "mistral:7b",
    "download_gb": 4.4
  },
  {
    "label": "qwen3:8b",
    "download_gb": 5.2
  },
  {
    "label": "gemma3:12b",
    "download_gb": 8.1
  },
  {
    "label": "qwen3:14b",
    "download_gb": 9.3
  },
  {
    "label": "gemma3:27b",
    "download_gb": 17
  },
  {
    "label": "qwen3:30b",
    "download_gb": 19
  }
]

All 7 rows na figures wey ollama.com/library publish, no be sizes wey dem measure for disk. The smallest tag here, gemma3:4b, dey download 3.3 GB. The biggest one, qwen3:30b, dey download 19 GB. The container image dey add on top of that for Podman own storage, so check both numbers together with podman system df and df -h /home. Model still need roughly the same RAM as the size of its file while e dey loaded, plus extra room for the context window. So, 19 GB model no go run for 16 GB VPS.

Pin image tag, and use full registry name

sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
mkdir -p ~/ollama-data ~/.config/containers/systemd
podman pull docker.io/ollama/ollama:0.32.9

Use released version tag, 0.32.9 as of August 2026, no be latest. Pinned tag mean say restart for 04:00 go give you the same binary wey you test, so any change for behaviour na change wey you make. Docker Hub still dey publish -rc and -rocm tags for the same versions; pick the plain one unless you get AMD GPU.

Write the registry host too. For Fedora, short name inside systemd unit no get terminal wey e fit prompt, and the unit go fail with:

Error: short-name "ollama/ollama" did not resolve to an alias and no unqualified-search registries are defined

Pulling am by hand first optional, but e useful because e move the multi-gigabyte download comot from the unit start timeout.

Quadlet unit wey go survive reboot

Quadlet na Podman systemd generator. You go write a .container file, systemd go turn am into service when system boot, and podman generate systemd no longer dey needed. Save am as /home/ollama/.config/containers/systemd/ollama.container, and make ollama user own am.

[Unit]
Description=Ollama API (rootless)
After=network-online.target
Wants=network-online.target

[Container]
Image=docker.io/ollama/ollama:0.32.9
ContainerName=ollama
PublishPort=127.0.0.1:11434:11434
Volume=/home/ollama/ollama-data:/root/.ollama:Z
Environment=OLLAMA_KEEP_ALIVE=30m
Environment=OLLAMA_MAX_LOADED_MODELS=1

[Service]
Restart=always
TimeoutStartSec=900

[Install]
WantedBy=default.target

The file name dey set the service name, so ollama.container go become ollama.service.

systemctl --user daemon-reload
systemctl --user start ollama.service
systemctl --user status ollama.service

status suppose show active (running). No run systemctl --user enable ollama.service. The unit no dey exist as file for disk, so systemd go reject am:

Failed to enable unit: Unit file /run/user/1001/systemd/generator/ollama.service is transient or generated.

The [Install] section don already handle that work. Quadlet go create the start-at-boot link by itself during daemon-reload, na why that command no be optional. TimeoutStartSec=900 dey cover the first start wey still need pull the image, because the default 90 seconds no reach for two-gigabyte download and systemd go kill the start as failed. OLLAMA_KEEP_ALIVE=30m dey keep one model for memory between requests instead of unloading am after five minutes; the trade-offs dey for keeping Ollama model loaded for memory. If any systemd vocabulary for here be new to you, how systemd services and timers dey work for VPS explain the units themselves.

Why model directory dey return permission denied under SELinux

For Fedora, RHEL, Rocky and AlmaLinux, SELinux dey enforcing by default. Container process dey run for container_t domain, while directory wey dey inside user home dey labelled user_home_t. Policy no allow one make changes to the other, so Ollama no fit create its model tree and container dey exit. getenforce dey print Enforcing for these systems, and dem record the denial:

sudo ausearch -m avc -ts recent

You go see one line wey name the domain and target label:

avc:  denied  { write } for  pid=1842 comm="ollama" name="models" dev="vda1" ino=131077 scontext=system_u:system_r:container_t:s0:c214,c827 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permlisted=0

The :Z for the end of Volume= line na the fix. E relabel the host directory to container_file_t and stamp am with private MCS (multi-category security) category wey na only this container carry. Lowercase :z use shared label instead, and na that one you want when two containers dey read the same directory.

One warning about :Z, because e destructive and quiet. Relabelling dey recurse. If you point am to /home/ollama, every file inside that home directory go get relabelled, and this go break SSH key access for that user. Always give :Z dedicated subdirectory wey no contain anything else. Named volumes no need am, because Podman dey label dem correctly when e create dem. If you need the bigger picture, SELinux basics for server explain contexts and booleans. For Ubuntu and Debian, na AppArmor dem dey use instead, :Z no-op there, and leaving am inside the unit no cause problem.

Close port 11434 and reach the API through SSH

PublishPort=127.0.0.1:11434:11434 dey bind the host side to loopback. Confirm am:

ss -ltnp | grep 11434
curl http://127.0.0.1:11434

The ss output must show 127.0.0.1:11434. 0.0.0.0:11434 or *:11434 means say the port dey open to internet, and the curl must answer Ollama is running.

Make you precise about the side wey you dey bind. The address for PublishPort na the host address. Inside the container, Ollama must continue listening on all interfaces, wey be the image default. If you set Environment=OLLAMA_HOST=127.0.0.1, e go bind Ollama to the container own loopback. Podman go forward published traffic to the container network address instead, so every request go dey refused, even from the host.

An open 11434 fit cause two problems. Ollama no get authentication, so anybody wey reach the port fit list your models through /api/tags, run inference on your CPU and bandwidth allowance through /api/generate, pull new models enter your disk, and delete the ones wey you get. Second, plain HTTP to a remote port dey send prompts and completions as cleartext, so every machine along the path fit read dem. Both problems go disappear if the port no ever comot from the box.

From your workstation, forward the port through SSH:

ssh -N -L 11434:127.0.0.1:11434 you@vps.example.com

Now http://127.0.0.1:11434 for your laptop na the server Ollama, inside the SSH session encryption. If your laptop already dey run Ollama, the local bind go fail with bind [127.0.0.1]:11434: Address already in use. Use -L 11435:127.0.0.1:11434 and point your client to 11435.

When browser client need am, put reverse proxy with password for front instead. A Caddy site block na four lines, and caddy hash-password go print the bcrypt hash wey e need:

ollama.example.com {
  basic_auth {
    you $2a$14$replace_with_the_generated_hash
  }
  reverse_proxy 127.0.0.1:11434
}

Caddy go get certificate through TLS (transport layer security) by itself, so the traffic dey encrypted. Test your client first: plenty tools wey dey talk to Ollama no get field for an Authorization header, and dem go fail against basic auth with a bare 401 Unauthorized. SSH tunnel no get this problem, na why e be the default recommendation here.

Pull model and check the complete path

podman exec -it ollama ollama pull gemma3:4b
curl -s http://127.0.0.1:11434/api/tags
curl -s http://127.0.0.1:11434/api/generate -d '{"model":"gemma3:4b","prompt":"Reply with the single word: ready","stream":false}'
du -sh ~/ollama-data/models

/api/tags dey return JSON list wey contain gemma3:4b. /api/generate dey return JSON object wey get response field, after small wait while the weights load from disk. du suppose report number wey near the published download size. Then prove the part wey this whole guide dey about:

sudo reboot
# reconnect, then:
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user is-active ollama.service

active mean say lingering dey work, the [Install] section and daemon-reload all do wetin dem suppose do. inactive mean say one of the three dey missing.

Failure modes, plus the strings wey you go see

Container disappear after reboot. Check loginctl show-user ollama --property=Linger first, because if Linger=yes no dey, the user's systemd instance no go start during boot. If lingering dey on, the [Install] section no dey inside .container file, or you edit the file but no run systemctl --user daemon-reload.

Error: statfs /home/ollama/ollama-data: no such file or directory. The bind mount source must exist before the container starts. Podman no dey create host directories for you. Run mkdir -p ~/ollama-data as the ollama user.

Start fail after 90 seconds. journalctl --user -u ollama.service dey show Start operation timed out. Terminating. because image pull still dey run. Pull am by hand, or keep TimeoutStartSec=900.

Container start and exit. podman logs ollama and sudo ausearch -m avc -ts recent together go tell you whether na SELinux label cause am. An AVC wey name container_t and user_home_t means say :Z no dey.

Requests dey refuse from the host. curl: (7) Failed to connect to 127.0.0.1 port 11434: Connection refused with the service active usually mean say OLLAMA_HOST set to a loopback address inside the container. Remove that line.

Generation dey very slow, or container dey get killed. If GPU no dey, inference go run on CPU, and large model naturally dey slow. If container die during request and signal: killed dey inside the logs, na kernel out-of-memory killer cause am, so choose smaller tag from the chart above.

Update image wey you pin

Pinning mean say na you dey decide when update go happen; e no dey happen by itself. Edit Image= inside ollama.container, then reload and restart:

systemctl --user daemon-reload
systemctl --user restart ollama.service
podman exec ollama ollama --version

Models dey inside bind mount, so image change no go affect dem. AutoUpdate=registry for the [Container] section na for people wey dey run moving tag. E no get useful work to do beside fixed version tag, because contents for that tag no dey change. Back up /home/ollama/ollama-data/models/manifests and .container file, and leave the blobs: dem big, and ollama pull go fetch dem again for new box.

FAQ

Why container wey use rootless Podman dey stop when I log out?

systemd instance for user and im /run/user/<uid> directory dey tear down when the last session for that user end, and every rootless container go end with dem. Run sudo loginctl enable-linger ollama and confirm say loginctl show-user ollama --property=Linger print Linger=yes. Enable lingering before you create the Quadlet unit, because runtime directory wey the unit need dey exist only after lingering don turn on.

I need SELinux labels for Ollama model directory?

For Fedora, RHEL, Rocky and AlmaLinux, yes, if you bind mount host directory. The container dey run for container_t domain, while directory inside home folder get user_home_t label, so write operation go fail and Ollama go exit. Append :Z to the Volume= line and give am dedicated subdirectory, because relabelling dey recurse and pointing :Z to whole home directory fit break SSH key access for that user. Podman dey label named volumes correctly, so dem no need anything extra.

How much disk space Ollama model need?

Start with published download size for ollama.com/library. E dey range from 3.3 GB for gemma3:4b reach 19 GB for qwen3:30b. Add Podman image on top, then leave extra space, because second model no go replace the first one for disk. Check df -h /home before you pull and du -sh ~/ollama-data/models after. Plan RAM the same way: model need roughly the same memory as im file size while e dey loaded, plus the context window.

E safe to expose port 11434 for VPS?

No. Ollama ships with no authentication at all, so anybody wey fit reach the port fit list your models, delete dem, pull new ones enter your disk, and run inference with your CPU and bandwidth allowance. Plain HTTP over internet also sends every prompt and completion as cleartext. Bind the host side to 127.0.0.1 with PublishPort=127.0.0.1:11434:11434, confirm with ss -ltnp | grep 11434, and reach am through SSH tunnel or reverse proxy wey require password.