Run Ollama with Rootless Podman for VPS

Rootless Podman for VPS run Ollama

For server wey you wan run Ollama with rootless Podman, five things must dey correct wey desktop walkthrough fit skip. One dedicated unprivileged user must own the container. You must enable lingering for that user, so container continue dey run after you log out. Quadlet file go hand the container over to systemd, so e go come back after reboot. Model directory must get correct SELinux label for distributions wey enforce am. API go listen only on loopback, and you go reach am through SSH (secure shell) tunnel.

Ollama na server for large language models (LLM). E dey store model weights for disk, load dem into memory, and answer HTTP requests for port 11434. E no get login, API key, or user accounts, so na network be the only access control wey you get. Podman dey run containers without daemon and without root, so anything wey escape from container go first start as ordinary unprivileged user. If you wan see runtime comparison first, read how Podman and Docker dey differ for VPS. If you prefer skip containers completely, how to install Ollama directly for VPS na shorter path.

SSD Nodes dey provision Fedora among its images, and Fedora dey ship both Podman and SELinux (security-enhanced Linux) by default. Every command below go run for any distribution wey get Podman 5 or newer.

Wetin make the laptop version need changes for server

Fedora Magazine publish one clear guide for this stack on 5 August 2026: Running Ollama Locally with Podman on Fedora Linux, wey Yazan Monshed write. E good as first one-hour introduction to the tools. But e target laptop, and four of the choices dey behave differently for machine wey get public IP address.

• E start the container with plain podman run -d. Container wey you start by hand no go come back after reboot, because nobody ever configure am to start.
• E use moving tag ollama/ollama. For laptop, you go notice the day wey the behaviour change. For server, the first sign fit be script wey stop working overnight.
• E publish with -p 11434:11434, wey bind every interface. Behind home router, internet no fit reach am. For VPS, e become public inference API without password.
• E run as your own login user. For server, the account wey own the container suppose own nothing else, so if break-out happen, e go land for empty home directory.

None of this wrong for the machine wey the guide target. Each item na decision wey you need review when the box dey reachable from everywhere and nobody dey sit in front of am.

Create the unprivileged user, and check subuid

Create the unprivileged user, and check subuid

Rootless Podman dey map the container internal user IDs (UID) go one block of unused IDs for the host. Dem declare this block for /etc/subuid and /etc/subgid. If this block no dey, rootless containers no fit start at all.

sudo dnf install -y podman        # or: sudo apt install -y podman
sudo useradd --create-home --shell /bin/bash --comment "Ollama container owner" ollama
sudo passwd --lock ollama
grep ollama /etc/subuid /etc/subgid

The grep suppose print two lines, one from each file, and each one go name a range of 65536 IDs:

/etc/subuid:ollama:100000:65536
/etc/subgid:ollama:100000:65536

Your starting number go different, and that one no be problem. If grep print nothing, useradd no allocate any range, and the first podman command wey you run as that user go fail like this:

Error: cannot find UID/GID for user ollama: no subuid ranges found for user "ollama" in /etc/subuid

Assign one range wey no other user dey use, then tell Podman say the old mapping don stale:

sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 ollama
sudo -iu ollama podman system migrate

Locking the password mean say nobody fit log in directly as ollama. You go reach the account from your admin user with sudo -iu ollama.

Enable lingering make service survive logout

A user get systemd instance wey normally start when dem login and stop when dem logout, and /run/user/<uid> dey remove together with am. Every rootless container wey that user own go die for the same time. Lingering keep the user instance running even when no session attach to am.

sudo loginctl enable-linger ollama
loginctl show-user ollama --property=Linger

That one suppose print Linger=yes. Enable am before you create the unit, because the directory wey the unit need, /run/user/<uid>, only dey exist after lingering don turn on.

One more step dey wey many people no expect. sudo -iu ollama give you shell but e no give you session bus, so systemctl --user fail immediately:

Failed to connect to bus: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not defined

systemd dey look for the user bus for $XDG_RUNTIME_DIR/bus, and sudo -i no set that variable. Set am by hand for every admin shell wey you use to manage this service:

sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user status

Wia model blobs dey land, and how much disk you suppose plan

Ollama dey write weights inside /root/.ollama/models for the container. Bind one directory from user home go that path, so files go land for place wey you fit measure: /home/ollama/ollama-data/models. Blobs dey inside models/blobs as content-addressed files, while models/manifests hold the small index wey dey name dem. If you use named volume instead, like Fedora Magazine post do, the same tree dey under /home/ollama/.local/share/containers/storage/volumes/<volume>/_data. Either way, ollama pull and ollama run dey write weights into the same tree. The only difference between the two commands na whether chat session go open after download finish.

Measure the disk before you pull anything. Published download sizes give you the minimum space wey you need.

[
  {
    "label": "gemma3:4b",
    "download_gb": 3.3
  },
  {
    "label": "mistral:7b",
    "download_gb": 4.4
  },
  {
    "label": "qwen3:8b",
    "download_gb": 5.2
  },
  {
    "label": "gemma3:12b",
    "download_gb": 8.1
  },
  {
    "label": "qwen3:14b",
    "download_gb": 9.3
  },
  {
    "label": "gemma3:27b",
    "download_gb": 17
  },
  {
    "label": "qwen3:30b",
    "download_gb": 19
  }
]

All 7 rows na figures wey ollama.com/library publish, no be sizes wey dem measure for disk. The smallest tag here, gemma3:4b, dey download 3.3 GB. The biggest one, qwen3:30b, dey download 19 GB. The container image itself dey use additional space for Podman own storage, so check both numbers together with podman system df and df -h /home. Model still need roughly the same amount of RAM as the file size while e dey loaded, plus extra space for context window. So, 19 GB model no go run for 16 GB VPS.

Pin image tag, and use full registry name

sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
mkdir -p ~/ollama-data ~/.config/containers/systemd
podman pull docker.io/ollama/ollama:0.32.9

Use released version tag, 0.32.9 as of August 2026, no be latest. Pinned tag mean say when restart happen for 04:00, na the same binary wey you test go run. So, if behaviour change, na change wey you make cause am. Docker Hub still publishes -rc and -rocm tags for the same versions. Pick the plain one unless you get AMD GPU.

Write the registry host too. For Fedora, short name inside systemd unit no get terminal wey e fit prompt. Because of this, the unit go fail with:

Error: short-name "ollama/ollama" did not resolve to an alias and no unqualified-search registries are defined

Pull by hand first no compulsory, but e useful. E moves the multi-gigabyte download comot from the unit start timeout.

Quadlet unit wey go survive reboot

Quadlet na Podman systemd generator. You go write one .container file, systemd go turn am into service during boot, and podman generate systemd no longer dey needed. Save am as /home/ollama/.config/containers/systemd/ollama.container, and make ollama user own am.

[Unit]
Description=Ollama API (rootless)
After=network-online.target
Wants=network-online.target

[Container]
Image=docker.io/ollama/ollama:0.32.9
ContainerName=ollama
PublishPort=127.0.0.1:11434:11434
Volume=/home/ollama/ollama-data:/root/.ollama:Z
Environment=OLLAMA_KEEP_ALIVE=30m
Environment=OLLAMA_MAX_LOADED_MODELS=1

[Service]
Restart=always
TimeoutStartSec=900

[Install]
WantedBy=default.target

The file name dey set service name, so ollama.container go become ollama.service.

systemctl --user daemon-reload
systemctl --user start ollama.service
systemctl --user status ollama.service

status suppose show active (running). No run systemctl --user enable ollama.service. The unit no dey exist as file for disk, so systemd go refuse am:

Failed to enable unit: Unit file /run/user/1001/systemd/generator/ollama.service is transient or generated.

The [Install] section don already handle that work. Quadlet go create the start-at-boot link by itself during daemon-reload, na why that command no be optional. TimeoutStartSec=900 dey cover first start wey still need pull the image, because the default 90 seconds no reach for two-gigabyte download and systemd go kill the start as failed. OLLAMA_KEEP_ALIVE=30m keep model for memory between requests instead of unloading am after five minutes; the trade-offs dey for keeping Ollama model loaded for memory. If any systemd vocabulary for here still new to you, how systemd services and timers dey work for VPS explain the units themselves.

Why SELinux dey return permission denied for model directory

For Fedora, RHEL, Rocky and AlmaLinux, SELinux dey enforcing by default. Container process dey run for container_t domain, while directory for user home dey labelled user_home_t. Policy no allow one touch the other, so Ollama no fit create its model tree and container go exit. getenforce dey print Enforcing for these systems, and denial dey recorded:

sudo ausearch -m avc -ts recent

You go see line wey name the domain and target label:

avc:  denied  { write } for  pid=1842 comm="ollama" name="models" dev="vda1" ino=131077 scontext=system_u:system_r:container_t:s0:c214,c827 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permlisted=0

The :Z for the end of Volume= line na the fix. E dey relabel host directory to container_file_t and stamp am with private MCS (multi-category security) category wey na only this container carry. Lowercase :z dey use shared label instead. Na this one you want when two containers dey read the same directory.

One warning about :Z, because e destructive and quiet. Relabelling dey recurse. If you point am to /home/ollama, every file for that home directory go get relabelled, and this go break SSH key access for that user. Always give :Z dedicated subdirectory wey no get anything else inside. Named volumes no need am, because Podman dey label dem correctly when e create dem. If you need the wider picture, SELinux basics for server dey explain contexts and booleans. For Ubuntu and Debian, na AppArmor dem dey use instead; :Z na no-op there, and leaving am inside the unit no cause problem.

Port 11434 close ampa reach the API through SSH

PublishPort=127.0.0.1:11434:11434 go bind the host side to loopback. Confirm am:

ss -ltnp | grep 11434
curl http://127.0.0.1:11434

The ss output suppose show 127.0.0.1:11434. 0.0.0.0:11434 or *:11434 mean say the port dey open to internet, and the curl suppose answer Ollama is running.

Make you precise about which side you dey bind. The address for PublishPort na the host address. Inside the container, Ollama must continue listening on all interfaces, wey be the image default. If you set Environment=OLLAMA_HOST=127.0.0.1, e go bind Ollama to the container own loopback. Podman go forward published traffic to the container network address instead, so every request go dey refused, even from the host.

Open 11434 fit cost you for two ways. Ollama no get authentication, so anybody wey reach the port fit list your models through /api/tags, run inference with your CPU and bandwidth allowance through /api/generate, pull new models enter your disk, and delete the ones wey you get. Second, plain HTTP to a remote port dey send prompts and completions as cleartext, so every machine along the path fit read dem. Both problems go disappear if the port no ever leave the box.

From your workstation, forward the port through SSH:

ssh -N -L 11434:127.0.0.1:11434 you@vps.example.com

Now http://127.0.0.1:11434 for your laptop na the server Ollama, inside the SSH session encryption. If your laptop already dey run Ollama, the local bind go fail with bind [127.0.0.1]:11434: Address already in use. Use -L 11435:127.0.0.1:11434 and point your client to 11435.

When browser client need am, put reverse proxy with password for front instead. Caddy site block na four lines, and caddy hash-password go print the bcrypt hash wey e need:

ollama.example.com {
  basic_auth {
    you $2a$14$replace_with_the_generated_hash
  }
  reverse_proxy 127.0.0.1:11434
}

Caddy go get certificate through TLS (transport layer security) by itself, so the traffic dey encrypted. Test your client first: plenty tools wey dey talk to Ollama no get field for Authorization header, and dem go fail against basic auth with bare 401 Unauthorized. SSH tunnel no get this problem, na why e be the default recommendation here.

Pull model come check the complete path

podman exec -it ollama ollama pull gemma3:4b
curl -s http://127.0.0.1:11434/api/tags
curl -s http://127.0.0.1:11434/api/generate -d '{"model":"gemma3:4b","prompt":"Reply with the single word: ready","stream":false}'
du -sh ~/ollama-data/models

/api/tags dey return JSON wey list gemma3:4b. /api/generate dey return JSON object wey get response field, after e pause while the weights dey load from disk. du suppose report number wey near the published download size. Then prove the part wey this whole guide dey talk about:

sudo reboot
# reconnect, then:
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user is-active ollama.service

active mean say lingering dey work, the [Install] section and daemon-reload all do their work. inactive mean say one of the three dey miss.

Wahala cases, plus the strings wey you go see

Container disappear after reboot. Check loginctl show-user ollama --property=Linger first, because if Linger=yes no dey, the user's systemd instance no go start during boot. If lingering dey on, the [Install] section no dey inside the .container file, or you edit the file but no run systemctl --user daemon-reload.

Error: statfs /home/ollama/ollama-data: no such file or directory. The bind mount source must dey before the container start. Podman no dey create host directories for you. Run mkdir -p ~/ollama-data as the ollama user.

Start fail after 90 seconds. journalctl --user -u ollama.service show Start operation timed out. Terminating. because the image pull still dey run. Pull am by hand, or keep TimeoutStartSec=900.

Container start and exit. podman logs ollama and sudo ausearch -m avc -ts recent together tell you whether na SELinux label cause am. An AVC wey name container_t and user_home_t mean say :Z no dey.

Requests dey get refused from the host. curl: (7) Failed to connect to 127.0.0.1 port 11434: Connection refused with the service active usually mean say OLLAMA_HOST set to a loopback address inside the container. Remove that line.

Generation dey very slow, or the container get killed. If GPU no dey, inference run for the CPU and large model naturally slow. If container die while request dey run and signal: killed dey inside the logs, na the kernel out-of-memory killer cause am, so choose a smaller tag from the chart above.

Update image wey you pin

Pinning mean say na you dey decide when update go happen, instead make update just happen by itself. Edit Image= for ollama.container, then reload and restart:

systemctl --user daemon-reload
systemctl --user restart ollama.service
podman exec ollama ollama --version

Models dey inside bind mount, so image change no go affect dem. AutoUpdate=registry for [Container] section na for people wey dey use moving tag, and e no get useful work to do beside fixed version tag, because the contents of that tag no dey change. Back up /home/ollama/ollama-data/models/manifests and .container file, and no include the blobs: dem big, and ollama pull go fetch dem again for new box.

FAQ

User logout kor free rootless Podman container dey stop?

User's systemd instance and im /run/user/<uid> directory dey tear down when that user's last session end, and every rootless container dey go with dem. Run sudo loginctl enable-linger ollama and confirm say loginctl show-user ollama --property=Linger dey print Linger=yes. Enable lingering before you create the Quadlet unit, because the runtime directory wey the unit need only dey exist after lingering don turn on.

I need SELinux labels for Ollama model directory?

For Fedora, RHEL, Rocky and AlmaLinux, yes, if you bind mount a host directory. The container dey run for container_t domain, while directory inside home folder get user_home_t label, so the write dey denied and Ollama dey exit. Append :Z to the Volume= line and give am dedicated subdirectory, because relabelling dey recurse and pointing :Z to complete home directory fit break SSH key access for that user. Podman dey label named volumes correctly, so dem no need anything extra.

How much disk Ollama model need?

Start with the published download size for ollama.com/library. E dey range from 3.3 GB for gemma3:4b reach 19 GB for qwen3:30b. Add the Podman image on top, then leave extra space, because second model no dey replace the first one for disk. Check df -h /home before you pull and du -sh ~/ollama-data/models after. Plan RAM the same way: model need roughly the size of im file for memory while e dey loaded, plus the context window.

E safe to expose port 11434 for VPS?

No. Ollama ships with no authentication at all, so anybody wey fit reach the port fit list your models, delete dem, pull new ones enter your disk, and run inference on your CPU and bandwidth allowance. Plain HTTP over internet also dey send every prompt and completion as cleartext. Bind the host side to 127.0.0.1 with PublishPort=127.0.0.1:11434:11434, confirm am with ss -ltnp | grep 11434, then reach am through SSH tunnel or reverse proxy wey require password.