Wetin PUID and PGID Do for Docker Compose

Wetin PUID and PGID really be

PUID and PGID na two environment variables wey some container images dey read when dem start. Docker itself no dey check dem. Na convention wey linuxserver.io images and some other images dey use. So, if image no dey written to read dem, e go silently ignore dem.

Inside linuxserver.io image, one user dey wey dem call abc. Dem create am during build time with UID (user ID) 911 and GID (group ID) 911. Container dey start as root, run im init scripts, and one of the scripts change the user ID before anything else happen:

groupmod -o -g "${PGID}" abc
usermod -o -u "${PUID}" abc

The -o flag allow ID wey already dey use somewhere else. After that, init drop privileges and run the application as abc. So PUID=1000 no ever reach Docker. The variable dey change the ID of one user inside the container before the application start. This mean say every file wey the application write for your disk go belong to 1000. If you leave PUID unset, abc go remain 911. Na why unconfigured bind mount dey fill with files wey belong to 911:911.

Get your two numbers with id

Run this for the host, as the user wey own the data directories:

id

uid=1000(deploy) gid=1000(deploy) groups=1000(deploy),27(sudo),988(docker)

uid na your PUID and gid na your PGID. For script, id -u and id -g go print only the numbers. For most fresh VPS images, the first human account na 1000:1000, but no assume say na so. If dem rebuild the server, or add another account later, e fit be 1001 or higher. Wrong number for here na the whole bug. If your services dey run under dedicated service account instead of your own login user, run id thatuser and use the numbers wey e show.

Wetin make your files show as 911:911

ls -l go print numeric ID instead of name when no account for host match that ID. No account for your server get UID 911, so e no get name to print. Use ls -ln to make am show numbers every time and remove the confusion:

ls -ln /srv/appdata/sonarr

drwxr-xr-x 2 911 911 4096 Aug  7 09:12 Backups
-rw-r--r-- 1 911 911  512 Aug  7 09:12 config.xml

That output show say the container run with the built-in defaults. Confirm am from inside the container instead of guessing:

docker exec sonarr id abc
docker compose logs sonarr | head -n 25

The linuxserver init go print the result for startup log as two lines:

User UID:    911
User GID:    911

If those lines show 911 after you set PUID=1000 for your Compose file, the variable no reach the container. The usual cause na say you edit docker-compose.yml and then run docker compose restart, wey reuse the existing container with the original environment. Environment changes need docker compose up -d, wey go recreate the container.

Why you no fit delete file wey container write

Kernel dey compare numbers, e no dey compare names. Your shell dey run as UID 1000. File belong to UID 911. Directory wey hold am na drwxr-xr-x and e too belong to 911, so group and other get read and execute but dem no get write. To delete file, you need write permission for its directory, not for the file, so na why you dey see this even when the file itself look harmless:

rm: cannot remove '/srv/appdata/sonarr/config.xml': Permission denied

Container wey dey write go hit the same problem from the other side. If host directory belong to your user with mode 755 and application dey run as 911, its first write go fail with Permission denied and app go report am with its own wording. For .NET application like Sonarr or Radarr, e go show as UnauthorizedAccessException: Access to the path '/data/downloads' is denied. Permission string wey dey before the file tell you which of the three permission sets system dey use judge you with, and how to read drwxr-xr-x correctly na wetin make that error change from confusing to obvious.

This one na bind mount problem specifically. When Docker create an empty named volume and mount am over path wey dey exist for image, e copy the path contents enter the volume, including ownership and permission bits, so application go find directory wey e already own. Bind mount no get this treatment: Docker mount your host directory exactly as e be. That difference na one practical reason why you need know when bind mount better pass named volume and when e no be better.

Fixing directory wey already get wrong settings

Setting PUID and PGID go change wetin application dey do from now. E no go retroactively fix files wey don dey disk already. Stop the stack, correct ownership by yourself, then start am again:

docker compose down
sudo chown -R 1000:1000 /srv/appdata/sonarr
docker compose up -d

Use sudo chown -R "$(id -u):$(id -g)" /srv/appdata/sonarr if you no wan type the numbers. Do this when container don stop, because application wey dey write at that time fit leave directory tree only partly corrected during recursive chown, and cause another confusing round of errors.

Wetin PUID and PGID no fit fix

Na this part dey catch people wey don do everything correct. The linuxserver init dey chown exactly three paths when e start: /app, /config and /defaults. Your media mounts no dey that list. /data, /downloads and /tv dey pass to the application untouched. So, if ownership for the host side of those mounts no allow the container user to write, the container go start cleanly, print the correct UID for its banner, then fail for the first import.

Na the correct behaviour be this. Recursive chown across a twelve terabyte media library every time container start go cause serious problem. E mean say na you dey responsible for the media directories. Na for those mounts permissions dey usually go wrong.

Ways wey you fit control user, and when each one apply

PUID and PGID environment variables

Dis one dey work only for images wey their entrypoint dey read dem. E popular because container still dey start as root, do e own setup, fix /config, then drop privileges. Docker Mods and custom init scripts still dey work. The cost na say you dey trust convention instead of platform feature, and variable names no standard across projects.

The user: key for Compose

Dis one na real Docker feature, and e dey work for every image because container runtime dey apply am before image own code run:

services:
  sonarr:
    image: lscr.io/linuxserver/sonarr:latest
    user: "1000:1000"

Process no dey run as root at all, not even for small time, and dis na real security gain. But e still dey break anything for entrypoint wey need root. For linuxserver images, project support dis on reasonable endeavours basis and only for images wey e don test. The caveats dey specific: PUID and PGID no go get any effect, Docker Mods no go run, custom services no go run, and na you go dey responsible for permissions on every mounted volume. Their documented pattern dey join the flag with writable /run:

user: 1000:1000
tmpfs:
  - /run:uid=1000,gid=1000,exec
security_opt:
  - no-new-privileges=true

One cosmetic side effect dey surprise people. Numeric user: no get matching entry for container own /etc/passwd, so tools inside dey report whoami: cannot find name for user ID 1000. The ID valid and file access dey work normally. Na only name lookup dey fail.

Rootless Docker

Rootless Docker dey run the daemon itself as your unprivileged user, so nothing for the box dey run as real root. E dey change the ownership arithmetic completely. Container UID 0 dey map to the host UID of the user wey dey run rootless Docker. Container UID n for any n of 1 or more dey map to subuid + (n - 1), where subuid na the base of the range wey dem allocate give you for /etc/subuid and /etc/subgid. Docker expect at least 65,536 subordinate IDs there.

Read dat mapping again because e dey turn the usual advice upside down. Under rootless Docker, container wey dey write as root go create files wey you own. Container wey dey write as UID 1000 go create files wey subordinate ID around 100999 own, and your shell no go fit touch dem. So PUID value wey correct for rootful daemon na wrong one here. Both mechanisms dey solve the same problem for different layers, and if you stack dem without checking, na so people dey end with directory wey dem need sudo to remove. If you choose rootless, test ownership of one file wey e write for your own server before you move library enter am.

For most self-hosted stacks for one VPS, PUID and PGID for rootful daemon na the practical choice because na wetin the images build and document for. Use user: when image README talk say dem test that image for am, or when you dey run official upstream image wey no get PUID support at all. Document workspace like self-hosted AFFiNE instance for one VPS dey for that second case, because none of the containers read PUID, and runtime dey decide ownership of database directory and uploaded files instead of anything for environment block.

The media stack case: one group wey containers share

An arr media stack wey get Sonarr, Radarr and one download client na where this matter stop to be theory. The download client dey write finished file inside /data/downloads. Sonarr then dey hardlink or move that file go /data/media. For hardlink to work, both containers need write access to the same tree. If download client dey run as 1000 while Sonarr dey run as 1001, one go own files wey the other fit only read.

The fix na shared group wey every container for the stack go use as its PGID:

sudo groupadd -g 13000 media
sudo usermod -aG media deploy
sudo chown -R deploy:media /srv/media
sudo find /srv/media -type d -exec chmod 2775 {} +
sudo find /srv/media -type f -exec chmod 0664 {} +

The leading 2 for 2775 na the setgid bit. For directory, e mean say every new file and subdirectory wey dem create inside go inherit group media instead of the creator own primary group. So this arrangement go continue to work for new downloads without you running chown again. Log out and log back in, or run newgrp media, before you check your own access. Group wey you add with usermod -aG no go show for shell session wey already open.

Inside the container, groupmod -o -g 13000 abc dey renumber the abc group to 13000. This make abc write with the same GID as your host media group. Every container for the stack keeps its own PUID and shares that one PGID.

Then set UMASK=002 for every linuxserver container inside the stack. Na this step people dey miss. The default for these images na UMASK=022. E dey remove group write bit from every new file, so files go land as 0644 and the sharing wey you just configure no go work. 002 dey produce 0664 files and 0775 directories, and the group fit write:

services:
  sonarr:
    image: lscr.io/linuxserver/sonarr:latest
    container_name: sonarr
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - UMASK=002
      - TZ=Etc/UTC
    volumes:
      - /srv/appdata/sonarr:/config
      - /srv/media:/data
    restart: unless-stopped

Put those two values inside a .env file beside the Compose file, so the whole stack go read one definition:

PUID=1000
PGID=13000

Compose dey read that file automatically for ${PUID} style substitution. Na the same mechanism you use for credentials. The practice of keeping values out of docker-compose.yml and putting dem inside a .env file apply here too, but these two numbers no be secret.

Verify everything end to end instead of trusting the config. Write one file from inside one container, then read am from the host:

docker exec sonarr touch /data/downloads/permtest
ls -ln /srv/media/downloads/permtest

A healthy result go show your PUID as the owner, 13000 as the group, and -rw-rw-r-- as the mode. If the group reads 1000, the setgid bit dey missing from that directory. If the mode reads -rw-r--r--, the UMASK variable no take effect. Check say you recreate the container instead of only restarting am. Remove the test file with rm /srv/media/downloads/permtest when you finish.

Which images dey use which variable

linuxserver.io images dey use PUID, PGID and UMASK. Paperless-ngx dey use different names for the same idea: USERMAP_UID and USERMAP_GID, and both default to 1000. Its documentation tell you make you read dem from id -u and id -g. Photo servers show the same difference: PhotoPrism get its own PHOTOPRISM_UID and PHOTOPRISM_GID pair, while Immich no ship any equivalent. Instead, e leave the container user to Docker user: key. So choosing between PhotoPrism and Immich also decide which of these mechanisms you go maintain for the biggest library for the server. Many official upstream images, including common database and web server images, ship with a fixed built-in user and expect you to use user: or leave am as e be. The same thing apply to infrastructure wey you add later. So putting Authentik in front of your apps for one login means running official server, Postgres and Redis images wey no read PUID at all. Their volume ownership come from the runtime, not from an entrypoint wey you fit configure.

So check the README for each image before you copy an environment block from one project go another. Docker pass any environment variable wey you set into any container, whether anything inside dey read am or not. A PUID wey nothing dey use no produce error, warning or effect. The container go run as whichever user its own Dockerfile finish with. You go discover this from the ownership of the files wey e write.

FAQ

Dem files for Docker dey show ownership as 911:911 because wetin?

911 na the UID and GID of the abc user wey dey inside linuxserver.io images. If you see am, e mean say container start without PUID and PGID set, so e init script leave the built-in defaults as dem be. ls -l dey show the raw numbers because no account for your host get ID 911, so no name dey to display. Set PUID and PGID to the output of id, recreate the container with docker compose up -d, then fix the existing files with sudo chown -R 1000:1000 for the affected directory.

usermod and groupmod dey work for every Docker image?

No. Dem no be Docker feature, and Docker no dey read dem. Dem only work for images wey their own entrypoint dey read dem and call usermod and groupmod before e start the application. Na linuxserver.io family and small number of projects wey copy this pattern dey do am. Other projects use different names, like USERMAP_UID and USERMAP_GID for paperless-ngx. For image wey no read any of dem, the variables go be accepted and ignored without warning.

I suppose use PUID and PGID or the user: key for Docker Compose?

Use PUID and PGID when the image support dem, because the entrypoint still dey run as root long enough to fix /config and start e own services correctly. Use user: when the image no support PUID, or when the image README talk say dem don test am for non-root operation. For linuxserver image, setting user: dey make PUID and PGID inactive, stop Docker Mods and custom services from running, and make permissions for every mounted volume your responsibility.

Sonarr get the correct PUID but e still no fit move files. Wetin dey wrong?

Check three things in order. First, check the media mount itself: the init only chowns /app, /config and /defaults, so /data or /downloads go keep whatever ownership e get for the host. Second, check the shared group: if the download client and Sonarr dey run with different GIDs, neither one fit modify the other one's files. So give every container for the stack the same PGID. Third, check the umask: the image default UMASK=022 dey write files as 0644 without group write bit, and this dey make shared group useless. Set UMASK=002 and set the setgid bit for the directories with chmod 2775 so new files inherit the group.