Python venv vs pipx vs uv on a server

Why pip install fails on a fresh Ubuntu server

Choosing between a Python venv, pipx and uv on a server comes down to one question: what are you installing? Application dependencies belong in a virtual environment inside the application's own directory. Command-line tools you want to type by name belong in pipx. uv does both jobs and adds a lockfile, which starts to matter as soon as a second machine has to build the same environment. What none of them do is install into the system Python, because a current Ubuntu server refuses that outright.

Run sudo pip install requests on Ubuntu 24.04 and pip stops before it downloads a single file.

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this behaviour by passing --break-system-packages.

This is PEP 668 (Python enhancement proposal 668, "externally managed environments") doing its job. Debian and Ubuntu drop a marker file beside the interpreter at /usr/lib/python3.12/EXTERNALLY-MANAGED, and pip refuses to write into any interpreter that carries one.

The rule exists because of sys.path order. apt installs libraries into /usr/lib/python3/dist-packages. pip, run as root against the system interpreter, writes into /usr/local/lib/python3.12/dist-packages, and Debian's packaging puts that directory earlier in the search path. Print it yourself with python3 -c 'import sys; print(sys.path)' and read the order. So the copy pip wrote shadows the copy apt installed, for every program on the box that runs under /usr/bin/python3, including the distribution's own tools. cloud-init imports requests, jinja2 and PyYAML from that interpreter. Upgrade one of those with pip, land on an incompatible release, and something you never touched fails at the next boot with a traceback naming a package you did not know was in the chain. apt still records its own version as installed, so nothing warns you, and the repair is sudo apt reinstall python3-requests.

The rule that follows is short. The system Python belongs to the distribution. Do not install into it, do not upgrade its libraries with pip, and do not delete the EXTERNALLY-MANAGED file to make the message go away. The one job to give /usr/bin/python3 is building virtual environments.

venv vs pipx vs uv: the decision rule

Pick by what you are installing, not by which tool you read about most recently.

• An application you deploy and run as a service, such as a Django or Flask project: one virtual environment (venv) inside that application's directory.
• A command-line tool you want on your PATH, such as ansible or httpie: pipx, which gives each tool a private environment and one link on PATH.
• A project that needs a lockfile, faster installs, or a Python version the distribution does not ship: uv, which produces an ordinary venv plus a uv.lock file.
• A library that a distribution tool needs rather than your code: sudo apt install python3-<name>, the only supported way to add anything to the system interpreter.

pipx and uv tool install do the same job, so a box that already has uv does not need pipx as well. Which web framework you chose changes nothing here: Django and Flask on a VPS differ in what lands in requirements.txt, not in how the environment around it is built. Everything below uses Ubuntu 24.04 and its Python 3.12, so adjust the version inside the paths if yours differs.

Build the per-application venv

Ubuntu splits the venv module out of the base Python package, so on a minimal image the first attempt fails with a message that names exactly what is missing.

The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.12-venv

Install it, then create the environment as the user that will own the code.

sudo apt update
sudo apt install -y python3-venv
sudo install -d -o deploy -g deploy -m 755 /srv/myapp
sudo -u deploy python3 -m venv /srv/myapp/.venv
sudo -u deploy /srv/myapp/.venv/bin/pip install -r /srv/myapp/requirements.txt

Notice what is not there: no source, no activate. /srv/myapp/.venv/bin/pip installs into that environment because of where the binary sits, not because of anything you exported into the shell. Confirm it before you go further.

/srv/myapp/.venv/bin/python -c 'import sys; print(sys.prefix)'

That prints /srv/myapp/.venv. If it prints /usr, you are running the system interpreter and your packages went somewhere you did not intend.

Two properties of a venv decide what you may do with it afterwards. A venv is not relocatable, because every script in bin/ carries an absolute shebang line: head -1 /srv/myapp/.venv/bin/pip reads #!/srv/myapp/.venv/bin/python. Rename the parent directory and those scripts fail with bad interpreter: No such file or directory. A venv also pins the interpreter that created it, recorded as the home line in /srv/myapp/.venv/pyvenv.cfg, and bin/python3 is a symlink to that binary. Upgrade the release so python3.12 is gone, the symlink has no target, and the service dies at start with No such file or directory. Both cases have the same fix: delete the venv and build a new one from requirements.txt. Rebuilding takes seconds. Never copy a venv between machines.

Where the venv lives, and who owns it

Put it beside the code at /srv/myapp/.venv, and keep one venv per application. The deployment is then a single directory, the systemd unit gets a path that never changes, and two applications can never break each other through one shared dependency upgrade. Do not place a venv anywhere your web server publishes files directly, since it holds your dependencies and often your configuration.

Ownership deserves half a minute. Let a deploy user own the code and the environment, and give the service account read and execute access only.

sudo adduser --system --group --no-create-home myapp
sudo chown -R deploy:myapp /srv/myapp
sudo chmod -R o-rwx /srv/myapp

The service can now import its dependencies but cannot rewrite them, which means a code execution bug in the web application cannot quietly replace a library on disk and survive a restart. The same reasoning applied to the rest of the machine is covered in running services as least-privilege users.

pipx for command-line tools

pipx installs applications, not libraries. Each tool gets its own environment under ~/.local/share/pipx/venvs/<name>, and that tool's executables are linked into ~/.local/bin, so two tools needing different versions of the same library never collide.

sudo apt update
sudo apt install -y pipx
pipx ensurepath
pipx install httpie

pipx ensurepath adds ~/.local/bin to PATH by editing your shell startup file. It cannot change the shell you are already sitting in, so http: command not found right after the install usually means you have not logged out and back in yet. Ubuntu's default ~/.profile adds ~/.local/bin only when that directory already exists at login, which is why this bites once on a new account and never again.

Point pipx at a library and it refuses, with a message that starts:

No apps associated with package requests or its dependencies.

That is the tool telling you it is the wrong instrument. Libraries belong in an application's venv.

The detail that matters on a server is location. A plain pipx install puts everything under one user's home directory. A systemd unit running as myapp cannot see it, a root cron job cannot see it, and sudo will not find it either, because secure_path in /etc/sudoers replaces PATH with a fixed list. For a tool the whole machine should have, install it globally.

sudo pipx install --global ansible
sudo pipx ensurepath --global

The --global flag places environments in /opt/pipx and links executables into /usr/local/bin, which is on the default PATH and inside secure_path. Check your version first with pipx --version, because Ubuntu 24.04 packages pipx 1.4.3, which is older than --global, and an older pipx answers with unrecognized arguments: --global. On that version, set the two documented directories yourself:

sudo env PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install ansible
command -v ansible

command -v ansible should print /usr/local/bin/ansible. If it prints a path under /home, the tool went into a single user's account and no service will find it.

uv when you want a lockfile

uv is a single binary from Astral that covers what pip, venv and pip-tools do, and it can download interpreters as well. It is fast enough that the difference shows on a small VPS, and it writes a real lockfile.

The official installer puts uv and uvx in ~/.local/bin:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv --version

Piping a script into a shell on a server deserves one moment of care. Pin the version in the URL and read the file before you run it:

curl -LsSf https://astral.sh/uv/0.12.3/install.sh -o uv-install.sh
less uv-install.sh
sh uv-install.sh

pipx install uv works too, when pipx is already there. uv is one self-contained binary with no Python dependency of its own, so copying it into /usr/local/bin is a valid way to share it with every user on the box.

For a project with a pyproject.toml, the workflow is four commands, and only the last one runs on the server.

uv init myapp
uv add flask gunicorn
uv lock
uv sync --frozen --no-dev

uv lock writes uv.lock, a cross-platform lockfile holding exact resolved versions, and you commit it next to your code. uv sync builds .venv in the project root to match it. On the server, --frozen is the flag that matters: the documentation defines it as using the versions in the lockfile as the source of truth instead of checking whether the lockfile is up to date, which is the behaviour a deployment wants. --no-dev leaves out the development dependency group.

An existing requirements.txt project needs no conversion, because uv speaks pip's language:

uv venv /srv/myapp/.venv
uv pip install --python /srv/myapp/.venv/bin/python -r /srv/myapp/requirements.txt

What comes out is an ordinary virtual environment. .venv/bin/python behaves exactly as it would if python3 -m venv had built it, so nothing later in this guide changes.

One uv default is worth knowing before you use it on a server. Its python-preference setting defaults to managed, documented as choosing "those that are downloaded and installed by uv" over interpreters already present on the system. So uv venv --python 3.13 on a box that ships only 3.12 quietly fetches 3.13 into ~/.local/share/uv/python rather than failing. That is convenient on a laptop and surprising on a server, because your service now depends on an interpreter that lives in a home directory and that apt upgrade will never patch. Set python-preference to only-system in uv.toml if you want the distribution's interpreter. If you want the environment somewhere other than the project root, UV_PROJECT_ENVIRONMENT specifies the directory to use for a project virtual environment.

Point systemd at the venv interpreter, not at activate

This is where most Python deployments break, and the cause is a misunderstanding of what activate does.

bin/activate is a shell script. It prepends the venv's bin directory to PATH, sets VIRTUAL_ENV, saves the old values so deactivate can restore them, and changes your prompt. It contains nothing that the interpreter itself reads. Activation is a convenience for a human typing python at a prompt.

What actually selects the environment is which interpreter file you execute. When /srv/myapp/.venv/bin/python starts, Python's site module looks for a pyvenv.cfg file in the directory holding the executable and one level above it. Finding /srv/myapp/.venv/pyvenv.cfg sets sys.prefix to the venv, which puts that venv's site-packages on sys.path. That is the entire mechanism. It needs no environment variable and no shell.

So this unit never starts:

[Service]
ExecStart=source /srv/myapp/.venv/bin/activate && gunicorn app:app

myapp.service: Failed to locate executable source: No such file or directory
myapp.service: Failed at step EXEC spawning source: No such file or directory
myapp.service: Main process exited, code=exited, status=203/EXEC

ExecStart is not a shell command line. systemd executes a program directly, so there is no source builtin, && is handed over as a literal argument, and nothing expands.

And this unit starts, then dies:

[Service]
ExecStart=/usr/bin/python3 /srv/myapp/app.py

ModuleNotFoundError: No module named 'flask'

/usr/bin/python3 is the system interpreter, and its sys.path has never contained your venv. The same command works in your SSH session only because you had activated the venv there, so the shell resolved python3 through PATH to .venv/bin/python3 instead.

Wrapping the command in /bin/bash -c 'source ... && gunicorn ...' does work. It also puts a shell between systemd and your process for no gain, when one absolute path settles it:

[Unit]
Description=myapp web service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/srv/myapp
Environment=PYTHONUNBUFFERED=1
Environment=PATH=/srv/myapp/.venv/bin:/usr/local/bin:/usr/bin:/bin
ExecStart=/srv/myapp/.venv/bin/gunicorn --workers 3 --bind 127.0.0.1:8000 app:app
Restart=on-failure
RestartSec=5
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=full

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now myapp
systemctl status myapp
journalctl -u myapp -n 50 --no-pager

systemctl status myapp should report active (running) with a Main PID that is your gunicorn process. Anything else, read the journal.

The Environment=PATH= line is not there for ExecStart, which already carries a full path. It is there for the processes your application starts. A service inherits a short default PATH from systemd, so Python code calling subprocess.run(["ffmpeg", ...]), or a management command that shells out to a console script from the venv, will not find what it needs. Putting the venv's bin directory first is the one part of activate that a service genuinely uses. Check what the unit really received with systemctl show -p Environment myapp.

The same rule covers scheduled work. cron runs jobs with a PATH of /usr/bin:/bin, so a crontab line reading python3 /srv/myapp/cleanup.py runs the system interpreter and fails with ModuleNotFoundError at three in the morning, and the error goes to a local mail spool nobody is reading. Write the absolute venv path there too. To get that output in the journal and a record of the last run instead, a systemd service and timer pair takes the same ExecStart line.

Does Docker replace this decision?

A container has a filesystem of its own, so the question changes shape rather than going away. In an official image such as python:3.12-slim, Python is built into /usr/local and carries no EXTERNALLY-MANAGED marker, so pip install as root is the intended way to add packages and a venv adds little. Build FROM ubuntu:24.04 instead and you meet externally-managed-environment again inside the image, for the same reason as on the host: it is the distribution's interpreter carrying the distribution's marker file.

Plenty of images still use a venv, because it makes a multi-stage build simple. The builder stage installs into /opt/venv, and the runtime stage copies that one directory and leaves the compilers behind. The activate problem travels with it. A RUN source /opt/venv/bin/activate line affects only that build layer's shell, so at runtime the container starts on the system interpreter and raises ModuleNotFoundError. Set ENV PATH="/opt/venv/bin:$PATH", or give CMD the absolute path /opt/venv/bin/gunicorn. It is the same bug as the systemd one, in a different file.

So a container replaces the interpreter question, because the image pins the interpreter and everything under it. It does not replace the pinning question. An image built from an unpinned requirements.txt resolves different versions next month, which means the image tag is reproducible while the build that produced it is not. A lockfile such as uv.lock, or a fully pinned requirements file, is what closes that gap, container or not. And when one application runs on one VPS under systemd, a container mostly moves this same decision into a Dockerfile, since systemd already restarts a failed process and captures its output in the journal. Running Docker on a VPS earns its keep when you want the built image itself to be the thing you deploy.

FAQ

Can I just use pip install with --break-system-packages?

Not on a server you have to keep running. The flag does what it says: it removes the guard, and pip writes into /usr/local/lib/python3.12/dist-packages, which comes before the apt directory on sys.path. Your version then shadows the distribution's for every system script running under /usr/bin/python3, and apt still believes its own version is installed, so nothing detects the conflict until something breaks. Inside a container image you rebuild from scratch each time, the damage stops at that image, so it is defensible there. On a machine you maintain, create a venv. That is one command.

Where should the virtual environment live on a server?

Inside the application's own directory, as /srv/myapp/.venv, owned by a deploy user, with the service account holding read and execute access only. Keep one venv per application, because a shared one means an upgrade for the first application can break the second. Do not move or copy a venv after creating it: every script in its bin/ directory has that absolute path written into its shebang line, so a moved venv fails with bad interpreter: No such file or directory. Delete it and rebuild from requirements.txt instead.

Why does my systemd service fail with ModuleNotFoundError?

Because the unit is executing an interpreter that is not the venv's. Run systemctl cat myapp and read ExecStart. It must name /srv/myapp/.venv/bin/python, or a console script from that same bin/ directory, by absolute path. Sourcing activate in a unit file cannot work, because ExecStart is not a shell, and systemd reports Failed to locate executable source with status=203/EXEC. Add Environment=PATH=/srv/myapp/.venv/bin:/usr/local/bin:/usr/bin:/bin so any subprocess your code starts finds the venv's tools too.

Should I use uv instead of venv and pip?

Use uv when you want a lockfile, when install time is slow enough to bother you, or when you need a Python version your distribution does not ship. It creates an ordinary venv, so the systemd unit and the file layout do not change, and uv sync --frozen installs exactly what the lockfile records. If a single application deploys from git with a pinned requirements.txt and the install finishes in seconds, python3 -m venv is already enough, and it is one less binary to keep updated on the server.