By Run Ollama in rootless Podman on a VPS
Run Ollama as a rootless Podman container on a VPS: a dedicated user, lingering, a Quadlet unit that survives reboot, SELinux labels, and a closed port.
Run Ollama in rootless Podman on a VPS
To run Ollama in rootless Podman on a server, five things have to be true that a desktop walkthrough can skip. A dedicated unprivileged user owns the container. Lingering is enabled for that user, so the container keeps running after you log out. A Quadlet file hands the container to systemd, so it comes back after a reboot. The model directory carries an SELinux label on distributions that enforce it. The API listens on loopback only, and you reach it through an SSH (secure shell) tunnel.
Ollama is a server for large language models (LLM). It stores model weights on disk, loads them into memory, and answers HTTP requests on port 11434. It has no login, no API key and no user accounts, so the network is the only access control you get. Podman runs containers with no daemon and with no root, so anything that escapes the container starts out as an ordinary unprivileged user. If you want the runtime comparison first, read how Podman and Docker differ on a VPS. If you would rather skip containers entirely, installing Ollama directly on a VPS is a shorter path.
SSD Nodes provisions Fedora among its images, and Fedora ships both Podman and SELinux (security-enhanced Linux) by default. Every command below runs on any distribution carrying Podman 5 or newer.
Why the laptop version needs changes on a server
Fedora Magazine published a clear walkthrough of this stack on 5 August 2026: Running Ollama Locally with Podman on Fedora Linux, by Yazan Monshed. It is a good first hour with the tools. It also targets a laptop, and four of its choices behave differently on a machine with a public IP address.
- It starts the container with a plain
podman run -d. A container started by hand does not come back after a reboot, because nothing was ever asked to start it. - It uses the moving tag
ollama/ollama. On a laptop you notice the day the behaviour changes. On a server the first sign is a script that stopped working overnight. - It publishes with
-p 11434:11434, which binds every interface. Behind a home router that is unreachable from the internet. On a VPS it is a public inference API with no password on it. - It runs as your own login user. On a server the account that owns the container should own nothing else, so a break-out lands in an empty home directory.
None of that is wrong for the machine it was written for. Each item is simply a decision you revisit when the box is reachable from everywhere and nobody is sitting in front of it.
Create the unprivileged user, and check subuid
Rootless Podman maps the container's internal user IDs (UID) onto a block of unused IDs on the host. That block is declared in /etc/subuid and /etc/subgid. Without it, rootless containers cannot start at all.
sudo dnf install -y podman # or: sudo apt install -y podman
sudo useradd --create-home --shell /bin/bash --comment "Ollama container owner" ollama
sudo passwd --lock ollama
grep ollama /etc/subuid /etc/subgidThe grep should print two lines, one from each file, each naming a range of 65536 IDs:
/etc/subuid:ollama:100000:65536
/etc/subgid:ollama:100000:65536Your starting number will differ, and that is fine. If the grep prints nothing, useradd did not allocate a range, and the first podman command as that user fails like this:
Error: cannot find UID/GID for user ollama: no subuid ranges found for user "ollama" in /etc/subuidAssign a range that no other user holds, then tell Podman its old mapping is stale:
sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 ollama
sudo -iu ollama podman system migrateLocking the password means nobody logs in as ollama directly. You reach the account from your admin user with sudo -iu ollama.
Enable lingering so the service survives logout
A user's systemd instance normally starts at login and stops at logout, and /run/user/<uid> is removed with it. Every rootless container owned by that user dies at the same moment. Lingering keeps the user instance running with no session attached.
sudo loginctl enable-linger ollama
loginctl show-user ollama --property=LingerThat should print Linger=yes. Enable it before you create the unit, because the directory the unit needs, /run/user/<uid>, only exists once lingering is on.
There is one more step nobody expects. sudo -iu ollama gives you a shell but not a session bus, so systemctl --user fails immediately:
Failed to connect to bus: $DBUS_SESSION_BUS_ADDRESS and $XDG_RUNTIME_DIR not definedsystemd looks for the user bus at $XDG_RUNTIME_DIR/bus, and sudo -i does not set that variable. Set it by hand in every admin shell where you manage this service:
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user statusWhere the model blobs land, and how much disk to plan
Ollama writes weights into /root/.ollama/models inside the container. Bind a directory from the user's home onto that path and the files land somewhere you can measure: /home/ollama/ollama-data/models. Blobs go in models/blobs as content-addressed files, and models/manifests holds the small index that names them. If you use a named volume instead, as the Fedora Magazine post does, the same tree lives under /home/ollama/.local/share/containers/storage/volumes/<volume>/_data.
Size the disk before you pull anything. Published download sizes give you the floor.
The data behind this chart
[
{
"label": "gemma3:4b",
"download_gb": 3.3
},
{
"label": "mistral:7b",
"download_gb": 4.4
},
{
"label": "qwen3:8b",
"download_gb": 5.2
},
{
"label": "gemma3:12b",
"download_gb": 8.1
},
{
"label": "qwen3:14b",
"download_gb": 9.3
},
{
"label": "gemma3:27b",
"download_gb": 17
},
{
"label": "qwen3:30b",
"download_gb": 19
}
]All 7 rows are figures published on ollama.com/library, not sizes measured on a disk. The smallest tag here, gemma3:4b, downloads 3.3 GB. The largest, qwen3:30b, downloads 19 GB. The container image itself sits on top of that in Podman's own storage, so check both numbers together with podman system df and df -h /home. A model also needs roughly its file size in RAM while it is loaded, plus room for the context window, so a 19 GB model on a 16 GB VPS will not run.
Pin the image tag, and use the full registry name
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
mkdir -p ~/ollama-data ~/.config/containers/systemd
podman pull docker.io/ollama/ollama:0.32.9Use a released version tag, 0.32.9 as of August 2026, and not latest. A pinned tag means a restart at 04:00 gives you the same binary you tested, so any change in behaviour is a change you made. Docker Hub also publishes -rc and -rocm tags for the same versions; pick the plain one unless you have an AMD GPU.
Write the registry host too. On Fedora a short name in a systemd unit has no terminal to prompt at, and the unit fails with:
Error: short-name "ollama/ollama" did not resolve to an alias and no unqualified-search registries are definedPulling by hand first is optional but useful, because it moves the multi-gigabyte download out of the unit's start timeout.
The Quadlet unit that survives a reboot
Quadlet is Podman's systemd generator. You write a .container file, systemd turns it into a service at boot, and podman generate systemd is no longer needed. Save this as /home/ollama/.config/containers/systemd/ollama.container, owned by the ollama user.
[Unit]
Description=Ollama API (rootless)
After=network-online.target
Wants=network-online.target
[Container]
Image=docker.io/ollama/ollama:0.32.9
ContainerName=ollama
PublishPort=127.0.0.1:11434:11434
Volume=/home/ollama/ollama-data:/root/.ollama:Z
Environment=OLLAMA_KEEP_ALIVE=30m
Environment=OLLAMA_MAX_LOADED_MODELS=1
[Service]
Restart=always
TimeoutStartSec=900
[Install]
WantedBy=default.targetThe file name sets the service name, so ollama.container becomes ollama.service.
systemctl --user daemon-reload
systemctl --user start ollama.service
systemctl --user status ollama.servicestatus should show active (running). Do not run systemctl --user enable ollama.service. The unit does not exist as a file on disk, so systemd refuses:
Failed to enable unit: Unit file /run/user/1001/systemd/generator/ollama.service is transient or generated.The [Install] section already does that job. Quadlet creates the start-at-boot link itself during daemon-reload, which is why that command is not optional. TimeoutStartSec=900 covers a first start that still has to pull the image, since the default 90 seconds is not enough for a two-gigabyte download and systemd will kill the start as failed. OLLAMA_KEEP_ALIVE=30m holds a model in memory between requests instead of unloading it after five minutes; the trade-offs are in keeping an Ollama model loaded in memory. If any of the systemd vocabulary here is new, how systemd services and timers work on a VPS covers the units themselves.
Why the model directory returns permission denied under SELinux
On Fedora, RHEL, Rocky and AlmaLinux, SELinux is enforcing by default. A container process runs in the container_t domain, and a directory in a user's home is labelled user_home_t. The policy does not let one touch the other, so Ollama cannot create its model tree and the container exits. getenforce prints Enforcing on these systems, and the denial is recorded:
sudo ausearch -m avc -ts recentYou will see a line naming the domain and the target label:
avc: denied { write } for pid=1842 comm="ollama" name="models" dev="vda1" ino=131077 scontext=system_u:system_r:container_t:s0:c214,c827 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permlisted=0The :Z at the end of the Volume= line is the fix. It relabels the host directory to container_file_t and stamps it with a private MCS (multi-category security) category that only this container carries. Lowercase :z uses a shared label instead, which is what you want when two containers read the same directory.
One warning about :Z, because it is destructive and quiet. Relabelling recurses. Point it at /home/ollama and every file in that home directory is relabelled, which breaks SSH key access for that user. Always give :Z a dedicated subdirectory that holds nothing else. Named volumes do not need it, because Podman labels those correctly when it creates them. If you need the wider picture, SELinux basics for a server explains contexts and booleans. On Ubuntu and Debian, AppArmor is used instead, :Z is a no-op there, and leaving it in the unit is harmless.
Close port 11434 and reach the API over SSH
PublishPort=127.0.0.1:11434:11434 binds the host side to loopback. Confirm it:
ss -ltnp | grep 11434
curl http://127.0.0.1:11434The ss output must show 127.0.0.1:11434. 0.0.0.0:11434 or *:11434 means the port is open to the internet, and the curl must answer Ollama is running.
Be precise about which side you are binding. The address in PublishPort is the host address. Inside the container, Ollama must keep listening on all interfaces, which is the image default. Setting Environment=OLLAMA_HOST=127.0.0.1 binds Ollama to the container's own loopback, and Podman forwards published traffic to the container's network address instead, so every request is refused even from the host.
An open 11434 costs you in two ways. Ollama has no authentication, so anyone who reaches the port can list your models through /api/tags, run inference on your CPU and your bandwidth allowance through /api/generate, pull new models onto your disk, and delete the ones you have. Second, plain HTTP to a remote port sends prompts and completions in cleartext, so every machine along the path can read them. Both problems disappear if the port never leaves the box.
From your workstation, forward the port over SSH:
ssh -N -L 11434:127.0.0.1:11434 you@vps.example.comNow http://127.0.0.1:11434 on your laptop is the server's Ollama, inside the SSH session's encryption. If your laptop already runs Ollama, the local bind fails with bind [127.0.0.1]:11434: Address already in use; use -L 11435:127.0.0.1:11434 and point your client at 11435.
When a browser client needs it, put a reverse proxy with a password in front instead. A Caddy site block is four lines, and caddy hash-password prints the bcrypt hash it wants:
ollama.example.com {
basic_auth {
you $2a$14$replace_with_the_generated_hash
}
reverse_proxy 127.0.0.1:11434
}Caddy gets a certificate over TLS (transport layer security) on its own, so the traffic is encrypted. Test your client first: many tools that talk to Ollama have no field for an Authorization header, and they will fail against basic auth with a bare 401 Unauthorized. The SSH tunnel has no such problem, which is why it is the default recommendation here.
Pull a model and check the whole path
podman exec -it ollama ollama pull gemma3:4b
curl -s http://127.0.0.1:11434/api/tags
curl -s http://127.0.0.1:11434/api/generate -d '{"model":"gemma3:4b","prompt":"Reply with the single word: ready","stream":false}'
du -sh ~/ollama-data/models/api/tags returns JSON listing gemma3:4b. /api/generate returns a JSON object with a response field, after a pause while the weights load from disk. du should report a number close to the published download size. Then prove the part that this whole guide is about:
sudo reboot
# reconnect, then:
sudo -iu ollama
export XDG_RUNTIME_DIR=/run/user/$(id -u)
systemctl --user is-active ollama.serviceactive means lingering, the [Install] section and daemon-reload all did their jobs. inactive means one of the three is missing.
Failure modes, with the strings you will see
Container is gone after a reboot. Check loginctl show-user ollama --property=Linger first, because without Linger=yes the user's systemd instance never starts at boot. If lingering is on, the [Install] section is missing from the .container file, or you edited the file and did not run systemctl --user daemon-reload.
Error: statfs /home/ollama/ollama-data: no such file or directory. The bind mount source must exist before the container starts. Podman does not create host directories for you. Run mkdir -p ~/ollama-data as the ollama user.
Start fails at 90 seconds. journalctl --user -u ollama.service shows Start operation timed out. Terminating. because the image pull was still running. Pull by hand, or keep TimeoutStartSec=900.
Container starts and exits. podman logs ollama and sudo ausearch -m avc -ts recent together tell you whether it is the SELinux label. An AVC naming container_t and user_home_t means the :Z is missing.
Requests are refused from the host. curl: (7) Failed to connect to 127.0.0.1 port 11434: Connection refused with the service active usually means OLLAMA_HOST was set to a loopback address inside the container. Remove that line.
Generation is very slow, or the container is killed. With no GPU, inference runs on the CPU and a large model is slow by nature. A container that dies mid-request with signal: killed in the logs is the kernel out-of-memory killer, so pick a smaller tag from the chart above.
Updating a pinned image
Pinning means updates are a thing you do, not a thing that happens to you. Edit Image= in ollama.container, then reload and restart:
systemctl --user daemon-reload
systemctl --user restart ollama.service
podman exec ollama ollama --versionModels live in the bind mount, so they survive the image change untouched. AutoUpdate=registry in the [Container] section exists for people running a moving tag, and it does nothing useful next to a fixed version tag, since that tag's contents never change. Back up /home/ollama/ollama-data/models/manifests and the .container file, and skip the blobs: they are large, and ollama pull fetches them again on a new box.
FAQ
Why does my rootless Podman container stop when I log out?
A user's systemd instance and its /run/user/<uid> directory are torn down when the last session for that user ends, and every rootless container goes with them. Run sudo loginctl enable-linger ollama and confirm loginctl show-user ollama --property=Linger prints Linger=yes. Enable lingering before you create the Quadlet unit, because the runtime directory the unit needs only exists once lingering is on.
Do I need SELinux labels on the Ollama model directory?
On Fedora, RHEL, Rocky and AlmaLinux, yes, if you bind mount a host directory. The container runs in the container_t domain and a directory in a home folder is labelled user_home_t, so the write is denied and Ollama exits. Append :Z to the Volume= line and give it a dedicated subdirectory, because relabelling recurses and pointing :Z at a whole home directory breaks SSH key access for that user. Named volumes are labelled correctly by Podman and need nothing extra.
How much disk does an Ollama model need?
Start from the published download size on ollama.com/library, which ranges from 3.3 GB for gemma3:4b up to 19 GB for qwen3:30b. Add the Podman image on top, then leave headroom, because a second model does not replace the first on disk. Check df -h /home before pulling and du -sh ~/ollama-data/models after. Plan RAM the same way: a model needs roughly its file size in memory while loaded, plus the context window.
Is it safe to expose port 11434 on a VPS?
No. Ollama ships with no authentication of any kind, so anyone who reaches the port can list your models, delete them, pull new ones onto your disk, and run inference on your CPU and your bandwidth allowance. Plain HTTP over the internet also sends every prompt and completion in cleartext. Bind the host side to 127.0.0.1 with PublishPort=127.0.0.1:11434:11434, confirm with ss -ltnp | grep 11434, and reach it through an SSH tunnel or a reverse proxy that requires a password.