Run Ollama with Rootless Podman for VPS
Run Ollama for VPS with rootless Podman: dedicated user, lingering, Quadlet after reboot, SELinux labels, and port 11434 locked to SSH tunnel.
Run Ollama for rootless Podman on VPS
To run Ollama for rootless Podman for server, five things must dey correct wey desktop walkthrough fit ignore. One dedicated unprivileged user must own the container. You must enable lingering for that user, so the container continue to run after you log out. A Quadlet file must hand the container to systemd, so e go come back after reboot. The model directory must get SELinux label for distributions wey enforce am. The API must listen for loopback only, and you go reach am through SSH (secure shell) tunnel.
Ollama na server for large language models (LLM). E dey store model weights for disk, load dem into memory, and answer HTTP requests for port 11434. Its local API no require login or API key, so na network be the only access control wey you get here. Podman dey run containers without daemon and without root, so anything wey escape from the container go first dey run as ordinary unprivileged user. If you want see runtime comparison first, read how Podman and Docker differ for VPS. If you prefer skip containers completely, installing Ollama directly for VPS na shorter way.
Fedora dey ship with both Podman and SELinux (security-enhanced Linux) by default. The commands below assume say na Podman 5 or newer you dey use, and the Quadlet unit still need cgroup v2, wey Podman documentation list as Quadlet requirement for any Podman version. Check both before you start: podman --version for the first one, and podman info | grep -i cgroup for the second one, where the cgroupVersion line must read v2. For Ubuntu 24.04, the archive's podman package na version 4.9.3, wey older pass that line, so check the version there first.
Wetin make the laptop version need changes for server
Fedora Magazine publish one clear guide for this stack on 5 August 2026: Running Ollama Locally with Podman on Fedora Linux, wey Yazan Monshed write. E good as first one-hour introduction to the tools. But e target laptop, and four of the choices dey behave differently for machine wey get public IP address.
- E start the container with plain
podman run -d. Container wey you start by hand no go come back after reboot, because nobody ever configure am to start. - E use moving tag
ollama/ollama. For laptop, you go notice the day wey the behaviour change. For server, the first sign fit be script wey stop working overnight. - E publish with
-p 11434:11434, wey bind every interface. Behind home router, internet no fit reach am. For VPS, e become public inference API without password. - E run as your own login user. For server, the account wey own the container suppose own nothing else, so if break-out happen, e go land for empty home directory.
None of this wrong for the machine wey the guide target. Each item na decision wey you need review when the box dey reachable from everywhere and nobody dey sit in front of am.
Create the unprivileged user, and check subuid
Rootless Podman dey map the container internal user IDs (UID) onto one block of unused IDs for the host. Dem declare this block for /etc/subuid and /etc/subgid. Podman support single-UID mode too, but configure subordinate ranges for this setup.
sudo dnf install -y podman # or: sudo apt install -y podman
sudo useradd --create-home --shell /bin/bash --comment "Ollama container owner" ollama
sudo passwd --lock ollama
grep ollama /etc/subuid /etc/subgidgrep suppose print two lines, one from each file, and each line go name one range of 65536 IDs:
/etc/subuid:ollama:100000:65536
/etc/subgid:ollama:100000:65536Your starting number go different, and that one no be problem. If grep print nothing, useradd no allocate any range. Podman go report missing mapping with message like this:
Error: cannot find UID/GID for user ollama: no subuid ranges found for user "ollama" in /etc/subuidAssign one range wey no other user dey hold, then tell Podman say the old mapping don stale:
sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 ollama
sudo -iu ollama podman system migrateWhen you lock the password, e block password login as ollama; e no block other authentication methods like SSH keys. You fit reach the account from your admin user with sudo -iu ollama.
Enable lingering make service survive logout
User's systemd instance normally dey start when dem login and stop after the last logout, and /run/user/<uid> dey remove together with am. Services wey that instance dey manage go stop when e stop. Lingering keep the user instance running even when no session dey attached.
sudo loginctl enable-linger ollama
loginctl show-user ollama --property=LingerThat one suppose print Linger=yes. Enable am before you create the unit, so the user manager and im runtime directory go dey available even without login session.
One more step dey wey many people no expect. sudo -iu ollama give you shell but e no give you session bus, so systemctl --user fail immediately:
Failed to connect to bus: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not definedsystemd dey look for the user bus for $XDG_RUNTIME_DIR/bus, but sudo -i no set that variable. Set am by hand for every admin shell wey you use to manage this service:
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user statusWhere model blobs go, and how much disk space you need plan
Ollama dey write weights inside /root/.ollama/models for the container. Bind one directory from user home to that path, so the files go somewhere wey you fit measure: /home/ollama/ollama-data/models. Blobs dey inside models/blobs as content-addressed files, while models/manifests hold the small index wey name dem. If you use named volume instead, as the Fedora Magazine post do, the same tree dey under /home/ollama/.local/share/containers/storage/volumes/<volume>/_data. Either way, ollama pull and ollama run write weights inside the same tree. The only difference between the two commands na whether chat session go open after download finish: wetin separate the two commands.
Measure the disk space before you pull anything. Published download sizes give you the minimum.
Di data wey dey behind dis chart
[
{
"label": "gemma3:4b",
"download_gb": 3.4
},
{
"label": "mistral:7b",
"download_gb": 4.4
},
{
"label": "qwen3:8b",
"download_gb": 5.2
},
{
"label": "gemma3:12b",
"download_gb": 8.2
},
{
"label": "qwen3:14b",
"download_gb": 9.3
},
{
"label": "gemma3:27b",
"download_gb": 17
},
{
"label": "qwen3:30b",
"download_gb": 19
}
]All 7 rows na figures wey ollama.com/library publish, not sizes wey dem measure for disk. The smallest tag here, gemma3:4b, downloads 3.4 GB. The biggest one, qwen3:30b, downloads 19 GB. The container image itself dey use extra space inside Podman storage, so check both numbers together with podman system df and df -h /home. For good performance, plan roughly the model file size in RAM, plus extra space for the context window. Physical RAM no be a hard limit: disk-backed memory mapping or enough swap fit allow 19 GB model to run for 16 GB VPS, but performance go reduce.
Pin image tag, and use full registry name
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
mkdir -p ~/ollama-data ~/.config/containers/systemd
podman pull docker.io/ollama/ollama:0.32.9Use released version tag, 0.32.9 for August 2026, no be latest. Newer releases don come out since then: for 30 October 2026, the newest plain version tag for Docker Hub na 0.40.13. So check the tags page and use the current release for both the pull command and the Image= line. Pinned version tag dey stop releases wey you no choose from entering the server. So, when behaviour change, na usually change wey you make. But tag still be pointer, e no be fingerprint: registry fit move am go another image anytime person push under the same name again. Only digest dey fixed. Podman accepts both forms for Image=. If restart for 04:00 must run exactly the binary wey you test, run podman images --digests docker.io/ollama/ollama after the pull and write Image=docker.io/ollama/ollama@sha256:<digest> instead. Docker Hub still publishes -rc and -rocm tags for the same versions. Pick the plain one unless you get AMD GPU.
Write the registry host too. For Fedora, short name inside systemd unit no get terminal wey e fit prompt. Because of that, the unit fails with:
Error: short-name "ollama/ollama" did not resolve to an alias and no unqualified-search registries are definedYou fit pull am by hand first, but e no compulsory. E useful because e moves the multi-gigabyte download comot from the unit start timeout.
Quadlet unit wey go survive reboot
Quadlet na Podman systemd generator. You go write one .container file, systemd go turn am into service during boot, and podman generate systemd no dey needed again. Save am as /home/ollama/.config/containers/systemd/ollama.container, and make ollama user own am.
[Unit]
Description=Ollama API (rootless)
After=network-online.target
Wants=network-online.target
[Container]
Image=docker.io/ollama/ollama:0.32.9
ContainerName=ollama
PublishPort=127.0.0.1:11434:11434
Volume=/home/ollama/ollama-data:/root/.ollama:Z
Environment=OLLAMA_KEEP_ALIVE=30m
Environment=OLLAMA_MAX_LOADED_MODELS=1
[Service]
Restart=always
TimeoutStartSec=900
[Install]
WantedBy=default.targetThe file name dey set the service name, so ollama.container go become ollama.service.
systemctl --user daemon-reload
systemctl --user start ollama.service
systemctl --user status ollama.servicestatus suppose show active (running). No run systemctl --user enable ollama.service. The unit no dey exist as file for disk, so systemd go reject am:
Failed to enable unit: Unit file /run/user/1001/systemd/generator/ollama.service is transient or generated.The [Install] section don already handle that work. Quadlet dey create the start-at-boot link by itself during daemon-reload, na why that command no be optional. TimeoutStartSec=900 dey cover the first start wey still need pull the image, because the default 90 seconds no reach for multi-gigabyte download and systemd go kill the start as failed. OLLAMA_KEEP_ALIVE=30m dey keep one model for memory between requests instead of unloading am after five minutes; the trade-offs dey for keeping Ollama model loaded for memory. If any systemd term here dey new to you, how systemd services and timers dey work for VPS explain the units themselves.
Why SELinux dey return permission denied for model directory
For Fedora, RHEL, Rocky and AlmaLinux, SELinux dey enforcing by default. Container process dey run for container_t domain, while directory for user home dey labelled user_home_t. Policy no allow one touch the other, so Ollama no fit create its model tree and container go exit. getenforce dey print Enforcing for these systems, and denial dey recorded:
sudo ausearch -m avc -ts recentYou go see line wey name the domain and target label:
avc: denied { write } for pid=1842 comm="ollama" name="models" dev="vda1" ino=131077 scontext=system_u:system_r:container_t:s0:c214,c827 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permlisted=0The :Z for the end of Volume= line na the fix. E dey relabel host directory to container_file_t and stamp am with private MCS (multi-category security) category wey na only this container carry. Lowercase :z dey use shared label instead. Na this one you want when two containers dey read the same directory.
One warning about :Z, because e destructive and quiet. Relabelling dey recurse. If you point am to /home/ollama, every file for that home directory go get relabelled, and this go break SSH key access for that user. Always give :Z dedicated subdirectory wey no get anything else inside. Named volumes no need am, because Podman dey label dem correctly when e create dem. If you need the wider picture, SELinux basics for server dey explain contexts and booleans. For Ubuntu and Debian, na AppArmor dem dey use instead; :Z na no-op there, and leaving am inside the unit no cause problem.
Close port 11434 and reach the API over SSH
PublishPort=127.0.0.1:11434:11434 dey bind the host side to loopback. Confirm am:
ss -ltnp | grep 11434
curl http://127.0.0.1:11434The ss output must show 127.0.0.1:11434, and the curl must answer Ollama is running. If ss shows 0.0.0.0:11434 or *:11434 instead, e mean say the port dey listen on every interface. Whether internet fit reach am after that depend on your firewall rules and the network wey dey in front of the server, so wildcard listener no mean say the port dey exposed automatically. But no rely on that: Ubuntu documentation talk say ufw dey disabled by default at first, and firewall rule na another thing wey must remain correct. Fix the bind address so the port no go ever listen publicly from the beginning.
Be precise about which side you dey bind. The address for PublishPort na the host address. Inside the container, Ollama must continue listening on all interfaces, wey na the image default. If you set Environment=OLLAMA_HOST=127.0.0.1, e go bind Ollama to the container own loopback. Podman go forward published traffic to the container network address instead, so every request go dey refused, even from the host.
Open 11434 dey cost you for two ways. Ollama no get authentication, so anybody wey fit reach the port fit list your models through /api/tags, run inference on your CPU and bandwidth allowance through /api/generate, pull new models enter your disk, and delete the ones wey you get. Second, plain HTTP to remote port dey send prompts and completions as cleartext, so every machine along the path fit read dem. Both problems go disappear if the port no ever leave the machine.
From your workstation, forward the port through SSH:
ssh -N -L 11434:127.0.0.1:11434 you@vps.example.comNow http://127.0.0.1:11434 for your laptop na the server Ollama, inside the SSH session encryption. If your laptop already dey run Ollama, the local bind go fail with bind [127.0.0.1]:11434: Address already in use; use -L 11435:127.0.0.1:11434 and point your client to 11435.
When browser client need am, put reverse proxy with password for front instead. Caddy site block short, and caddy hash-password go print the bcrypt hash wey e need:
ollama.example.com {
basic_auth {
you $2a$14$replace_with_the_generated_hash
}
reverse_proxy 127.0.0.1:11434
}Caddy go get certificate through TLS (transport layer security) by itself, so the traffic go dey encrypted. Test your client first: clients wey no fit send Authorization header go fail against basic auth with bare 401 Unauthorized. SSH tunnel no get this problem, na why e be the default recommendation here.
Pull model and check the complete path
podman exec -it ollama ollama pull gemma3:4b
curl -s http://127.0.0.1:11434/api/tags
curl -s http://127.0.0.1:11434/api/generate -d '{"model":"gemma3:4b","prompt":"Reply with the single word: ready","stream":false}'
du -sh ~/ollama-data/models/api/tags dey return JSON wey list gemma3:4b. /api/generate dey return JSON object wey get response field, after e pause small while e load the weights from disk. du suppose report number wey near the published download size. Then prove the part wey this whole guide dey explain:
sudo reboot
# reconnect, then:
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user is-active ollama.serviceactive mean say lingering dey work, while the [Install] section and daemon-reload all do their work. If e report inactive, check those three first, then inspect journalctl --user -u ollama.service for startup errors.
Failure modes, and the strings wey you go see
Container don disappear after reboot. Check loginctl show-user ollama --property=Linger first, because if Linger=yes no dey, the user’s systemd instance no go start during boot. If lingering dey on, check the [Install] section for .container file, run systemctl --user daemon-reload, and inspect journalctl --user -u ollama.service for startup errors.
Error: statfs /home/ollama/ollama-data: no such file or directory. The bind mount source must exist before the container starts. Podman no dey create host directories for you. Run mkdir -p ~/ollama-data as the ollama user.
Start fails after 90 seconds. journalctl --user -u ollama.service shows Start operation timed out. Terminating. because image pull still dey run. Pull am by hand, or keep TimeoutStartSec=900.
Container starts and exits. podman logs ollama and sudo ausearch -m avc -ts recent together go tell you whether SELinux label cause the problem. An AVC wey name container_t and user_home_t means say :Z dey missing.
Requests from the host dey get refused. curl: (7) Failed to connect to 127.0.0.1 port 11434: Connection refused with service active usually means say OLLAMA_HOST set to loopback address inside the container. Remove that line.
Generation dey very slow, or the container dey get killed. If GPU no dey, inference go run on CPU, and large model naturally dey slow. Disk-backed memory mapping or swap go make inference slower when model no fit inside physical RAM. If container die during request and signal: killed dey inside logs, check the cause; if kernel log confirm say na out-of-memory kill, choose smaller tag from the chart above.
Updating a pinned image
Pinning mean say na you dey decide when update go happen, no be something wey just happen by itself. Edit Image= inside ollama.container, then reload and restart:
systemctl --user daemon-reload
systemctl --user restart ollama.service
podman exec ollama ollama --versionModels dey inside the bind mount, so image change no go affect dem. AutoUpdate=registry for the [Container] section dey configure digest checks against the registry; podman auto-update, whether you run am manually or through its systemd timer, go pull changed image and restart the unit. This one fit work for a moving tag. But beside a version tag, e go run only when publisher push that same tag again. Na exactly this kind unplanned change pinning dey prevent, so leave am off here. Back up /home/ollama/ollama-data/models/manifests and the .container file, but skip the blobs: dem big, and ollama pull go fetch dem again for a new box.
FAQ
Why rootless Podman container dey stop when I log out?
Normally, systemd instance for one user and e /run/user/<uid> directory dey tear down after the user’s last session end. This one stop services wey that instance dey manage. Run sudo loginctl enable-linger ollama and confirm say loginctl show-user ollama --property=Linger print Linger=yes. Enable lingering before you create the Quadlet unit, so the user manager and e runtime directory fit dey available without login session.
I need SELinux labels for Ollama model directory?
For Fedora, RHEL, Rocky and AlmaLinux, yes, if you bind mount host directory. The container dey run for container_t domain, while directory inside home folder get user_home_t label. Because of this, the write dey denied and Ollama dey exit. Add :Z to the Volume= line and give am dedicated subdirectory. Relabelling dey recurse, and if you point :Z to the whole home directory, e go break SSH key access for that user. Podman dey label named volumes correctly, so dem no need anything extra.
How much disk Ollama model need?
Start with the published download size for ollama.com/library. The size dey range from 3.4 GB for gemma3:4b reach 19 GB for qwen3:30b. Add the Podman image on top, then leave extra space, because second model no replace the first one for disk. Check df -h /home before you pull and du -sh ~/ollama-data/models after. For good performance, plan roughly the model file size in RAM, plus extra space for the context window. Disk-backed memory mapping or enough swap fit allow bigger model to run with less physical RAM, but performance go reduce.
E safe to expose port 11434 for VPS?
No. Ollama ships without any authentication, so anybody wey fit reach the port fit list your models, delete dem, pull new ones enter your disk, and run inference on your CPU and bandwidth allowance. Plain HTTP over internet also sends every prompt and completion as cleartext. Bind the host side to 127.0.0.1 with PublishPort=127.0.0.1:11434:11434, confirm with ss -ltnp | grep 11434, and reach am through SSH tunnel or reverse proxy wey require password.