systemd dependencies and conditions explained

Requires does not mean After

systemd dependencies and conditions are four separate mechanisms that most unit files use as if they were one. Requires= and Wants= decide which other units get pulled in. After= and Before= decide the order units start in. ExecStartPre= runs a check that can fail the unit. The Condition and Assert families decide whether the unit runs at all. Each mechanism is independent of the others, so a unit can require another unit and still start at the same instant as it.

That last sentence is the bug behind almost every "it works when I start it by hand, but it fails at boot" report.

[Unit]
Description=Inventory API
Requires=postgresql.service

[Service]
ExecStartPre=/usr/bin/pg_isready -h 127.0.0.1 -t 5
ExecStart=/usr/local/bin/inventory-api

Requires=postgresql.service pulls PostgreSQL into the same start transaction. It does not wait for it. systemd starts both jobs in parallel, so pg_isready runs while PostgreSQL is still opening its data directory. It exits 2 because nothing is listening yet, and the unit fails before ExecStart is ever reached. Running sudo systemctl start inventory-api an hour later works, because by then PostgreSQL is already up. Nothing in the unit file changed, which is why the file looks innocent.

The fix is one line.

[Unit]
Requires=postgresql.service
After=postgresql.service

A sharper detail hides in the same place. A failing Requires= dependency only stops your unit from starting when you also set After= on it. Without the ordering, systemd has already started your unit by the time the other one fails, so there is nothing left to cancel. Requires= on its own does not buy the protection people think they are buying. Write After= next to every Requires= and every Wants= unless you have a specific reason not to.

What Requires, Wants, Requisite and BindsTo promise

All of these are dependency settings. None of them order anything.

• Wants=: pull the other unit in. If it fails, or does not exist, this unit starts anyway. This is what systemctl enable creates, as a symlink inside a .wants/ directory.
• Requires=: pull the other unit in. If it fails and you also ordered After= it, this unit is not started. If the other unit is later stopped explicitly, this unit is stopped with it.
• Requisite=: do not pull the other unit in. If it is not already active, fail this unit immediately.
• BindsTo=: like Requires=, and this unit also stops whenever the other unit stops for any reason, including hardware that disappears.
• PartOf=: stop and restart propagate from the other unit down to this one. Start does not propagate.
• Conflicts=: starting this unit stops the other one.

For one daemon that talks to another daemon, Wants= plus After= is usually the right pair. Requires= couples the lifetimes: stop the database for maintenance and your application goes down with it, and it does not come back when the database returns. Wants= plus After= gives you the boot ordering without that coupling, and a restart policy handles the case where the dependency vanishes later.

You also inherit dependencies you never wrote. With DefaultDependencies=yes, which is the default, an ordinary service gets Requires=sysinit.target, After=sysinit.target basic.target and Conflicts=shutdown.target for free. That is why a service with an almost empty [Unit] section still starts late in boot and still gets stopped cleanly at shutdown.

After and Before order the transaction, nothing else

After= and Before= are pure ordering. They carry no requirement at all. After=redis.service in a unit that nothing else pulls Redis into is a no-op: if redis.service is not part of the transaction, there is nothing to wait for, so your unit starts immediately.

That is worth saying twice, because it is the exact shape of the network-online.target mistake further down. Ordering waits only for units that are already being started in the same transaction.

The pair is symmetric. After=b.service written in a.service means the same thing as Before=a.service written in b.service, so use one of them, and put it in the unit you own. Ordering is reversed automatically at shutdown, so After=b.service also means your unit is stopped before b.service is.

After= waits for "started", and Type= defines what that means

After= waits until the other unit has finished starting. What "finished starting" means is decided entirely by that unit's Type=.

• Type=simple: as soon as systemd has forked the process. The program may not have parsed its config yet, let alone opened a socket.
• Type=exec: as soon as execve() succeeded. Slightly stronger. Still says nothing about readiness.
• Type=forking: when the original parent process exits.
• Type=oneshot: when the process exits. Here "started" really does mean the work is finished.
• Type=notify: when the service sends READY=1 on its notification socket. This is the only type that reports genuine readiness.

So After= on a Type=simple daemon is a weak promise, and that is the second half of the race in the first example. If the unit you depend on ships as Type=simple, ordering after it does not mean it is accepting connections. Two honest answers. Order after its socket unit instead, so the kernel queues incoming connections while the daemon is still starting. Or make your own service retry and let the restart policy carry it. Which type a unit uses is visible in systemctl cat, and the Type= setting and what each value tells systemd is worth reading before you rely on ordering.

ExecStartPre is a gate that can fail the unit

ExecStartPre= runs before ExecStart=. If it exits non-zero, activation is aborted and the unit goes to failed. ExecStart= never runs. This is the mechanism behind a large share of units that fail with no message from the actual program, because the program was never started.

Facts that catch people:

• It is not a shell. No pipes, no redirection, no globs, no &&. The first token must be an absolute path. Wrap the line in /bin/sh -c '...' when you need shell syntax.
• A - prefix makes a non-zero exit non-fatal: ExecStartPre=-/usr/bin/optional-check.
• Every ExecStartPre= must exit before the next one runs. It cannot start a long-running process.
• All ExecStartPre= lines share TimeoutStartSec= with ExecStart=. A pre-check that loops waiting for a database eats the start timeout, and the unit then fails with Result: timeout after start operation timed out. Terminating. appears in the journal.

The failure line names the control process, not the main process:

inventory-api.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
inventory-api.service: Failed with result 'exit-code'.

Read that symbolic name carefully. systemd maps small exit codes through a fixed table, so 2 always prints as INVALIDARGUMENT whatever the program meant by it. status=203/EXEC is the one that carries real information: systemd could not execute the binary at all, because the path is wrong or the file is not executable.

Do not use ExecStartPre= to create directories. RuntimeDirectory=, StateDirectory=, LogsDirectory= and CacheDirectory= create them with the right owner and mode, and RuntimeDirectory= is cleaned up when the service stops. They also behave correctly under DynamicUser=, which a hand-written mkdir does not.

Condition fails quietly. Assert fails loudly.

The Condition and Assert families run the same tests. They differ only in what happens when a test fails.

A failed Condition...= skips the unit. The start job is reported as successful. The unit stays inactive (dead), nothing is marked failed, no alert fires, and the journal records one line:

Condition check resulted in Inventory API being skipped.

On systemd 250 and newer, systemctl status prints the reason directly:

     Active: inactive (dead)
  Condition: start condition unmet at Thu 2026-08-20 09:14:02 UTC; 2min ago

The indented line under it names the exact directive that failed, for example ConditionPathExists=/etc/inventory/api.conf was not met.

A failed Assert...= fails the unit. The journal says Assertion failed for Inventory API. and the unit ends in failed (Result: assert), which is loud enough for monitoring to notice.

Pick between them by asking what a failed test means. Condition means "this unit does not apply on this machine". Assert means "this must be true, and if it is not, tell someone". Most units want Condition. Reach for Assert only when doing nothing silently is worse than a failed unit.

Two traps come with the Condition family.

First, a failed condition does not fail the units that depend on it. If a.service has Requires=b.service and b.service skips on a condition, the start job for b.service still counts as done, so a.service starts normally into a world where b is not running. A condition protects only the unit it is written in.

Second, conditions are evaluated every time the unit starts, at the moment the job runs. A unit triggered by a systemd timer on a VPS can be skipped a hundred times in a row and never once look failed. That is the same class of silent no-op as a cron job that runs but does nothing, and you find it the same way: read the journal for the unit instead of trusting its exit state.

The conditions worth knowing on a server:

• ConditionPathExists=/etc/inventory/api.conf, and its negation ConditionPathExists=!/etc/inventory/api.conf.
• ConditionFileNotEmpty= and ConditionDirectoryNotEmpty=, for a config file or a data directory that a package created but left empty.
• ConditionVirtualization=, so a unit needing a real kernel interface can carry ConditionVirtualization=!container. Check what your box reports with systemd-detect-virt.
• ConditionHost= matches the hostname or the machine ID, which is how one shared unit file behaves differently on two servers.
• ConditionKernelCommandLine= and ConditionKernelVersion=, for units tied to a boot parameter or to a minimum kernel.

An empty assignment clears the list, which is how a drop-in removes a condition that a package shipped:

[Unit]
ConditionPathExists=
ConditionPathExists=/srv/inventory/api.conf

Why network.target does not mean the network is up

network.target is a synchronization point, not a state. At boot, ordering after it means the network management software has been started. It does not mean that an interface has an address, or that a route to the internet exists. The target mostly exists for the other direction: a unit ordered After=network.target is stopped before the network is torn down at shutdown.

network-online.target is the one that waits. It is backed by a wait-online service belonging to whichever network manager you run:

• systemd-networkd-wait-online.service when systemd-networkd manages the links, which is the normal case on an Ubuntu server configured through netplan.
• NetworkManager-wait-online.service under NetworkManager.

Older ifupdown setups get the same effect from networking.service instead. Whichever one you have, using the target correctly takes two lines, not one.

[Unit]
Wants=network-online.target
After=network-online.target

network-online.target is not part of the default boot transaction, and nothing pulls it in for you. Write only After= and you are ordering against a unit that was never queued, so the ordering does nothing at all. That is the no-op described earlier, in its most expensive form. The Wants= line is what puts the target into the transaction so the After= line has something to wait for.

The second thing to know is that "online" is defined by the wait-online implementation, not by systemd. systemd-networkd-wait-online returns once the links it manages reach a configured state. It does not check that DNS resolves, and it does not check that any remote host is reachable.

That definition produces a common VPS failure. A machine with a second interface for a private network, declared in netplan but never given an address, makes the wait service sit there until it gives up:

systemd-networkd-wait-online[612]: Timeout occurred while waiting for network connectivity.
systemd-networkd-wait-online.service: Failed with result 'exit-code'.

Boot takes two extra minutes because the default timeout is 120 seconds. Two fixes. Mark the unused interface optional: true in the netplan file, so networkd stops waiting for it. Or add a drop-in on the wait service that names the link you care about with --interface=, or that passes --any to return as soon as one link is up.

Better still, avoid needing the target. Many services are ordered after network-online.target only because they bind one specific address and fail at boot with a line like this:

nginx: [emerg] bind() to 203.0.113.10:443 failed (99: Cannot assign requested address)

The kernel refuses the bind because that address is not up yet. Setting net.ipv4.ip_nonlocal_bind=1 lets a process bind an address the box does not hold yet, and a restart policy covers the rest. Delaying the whole boot on network readiness is a heavy tool for a problem that is usually one socket.

How to read the real systemd dependencies on a running box

Never reason from the unit file alone. Drop-ins, .wants/ symlinks and implicit default dependencies all add edges the file does not show.

systemctl cat inventory-api.service

This prints the unit file and every drop-in, in the order they apply, with the source path above each block. Run it first. A five-line override in /etc/systemd/system/inventory-api.service.d/ beats the packaged file and is otherwise invisible.

systemctl show inventory-api.service -p Requires -p Wants -p After -p Before -p ConditionResult -p AssertResult

This prints resolved values, after drop-ins and after systemd added its implicit dependencies. ConditionResult=no is the direct answer to "the unit reported success and did nothing".

systemctl list-dependencies inventory-api.service
systemctl list-dependencies --reverse inventory-api.service
systemctl list-dependencies --after inventory-api.service
systemctl list-dependencies --before inventory-api.service

The plain form walks Requires= and Wants= downward. --reverse shows which units pull yours in, which is how you find the target that starts it at boot. --after and --before show ordering, and that is the pair to read when the question is whether anything actually waited.

journalctl -b -u inventory-api.service --no-pager
journalctl -b -o short-precise -u inventory-api.service -u postgresql.service

The second command interleaves two units with millisecond timestamps. That is how you prove an ordering race instead of guessing at it. The pg_isready failure lands before PostgreSQL logs database system is ready to accept connections, and the gap between them is right there in the output.

systemd-analyze verify /etc/systemd/system/inventory-api.service
systemd-analyze critical-chain inventory-api.service

verify loads the unit the way systemd would and reports unknown directives, dependencies on units that do not exist, ordering cycles, and syntax it cannot parse. It changes nothing on the system. critical-chain prints the ordering chain that delayed the unit, with the time each step became active, and it works only for a unit that started during the current boot.

After editing any unit file, run sudo systemctl daemon-reload. To change a packaged unit, use sudo systemctl edit inventory-api.service, which creates a drop-in for you. Editing the vendor file under /usr/lib/systemd/system/ works until the next package upgrade replaces it. The same drop-in mechanism is how you attach memory and CPU limits to a service without touching a file the package owns.

Ordering cycles, and the line they leave in the journal

Add ordering in both directions and systemd breaks the loop by deleting one of the jobs:

systemd[1]: Found ordering cycle on inventory-api.service/start
systemd[1]: Job postgresql.service/start deleted to break ordering cycle starting with inventory-api.service/start

systemd picks which job to delete, and it may not pick the one you would. The result presents as a service that is missing after some reboots and present after others, which is miserable to debug from the outside. Most cycles come from units that set DefaultDependencies=no and then order themselves against basic.target anyway, or from adding Before= to a unit that already had After= pointing back at you. systemd-analyze verify finds them without a reboot.

The fixed unit

[Unit]
Description=Inventory API
Wants=postgresql.service network-online.target
After=postgresql.service network-online.target
ConditionPathExists=/etc/inventory/api.conf

[Service]
Type=notify
StateDirectory=inventory
ExecStart=/usr/local/bin/inventory-api
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

Every line does one job. Wants= pulls both dependencies into the transaction without tying this unit's lifetime to them. After= does the waiting, and it has to repeat both names because dependency and ordering are separate settings. ConditionPathExists= means a machine that has the package but not the config skips the unit quietly instead of alarming, which is the right behaviour for a config-driven service. Type=notify means anything ordered after this unit waits for real readiness rather than for a fork. Restart=on-failure covers the database going away long after boot, because ordering only applies to the first start. How aggressive that retry should be is what the Restart= and RestartSec= settings control.

Check it before you trust it:

sudo systemctl daemon-reload
systemd-analyze verify /etc/systemd/system/inventory-api.service
systemctl list-dependencies --after inventory-api.service
sudo systemctl start inventory-api.service
systemctl show inventory-api.service -p ConditionResult -p ActiveState -p Result

A healthy unit reads ConditionResult=yes with ActiveState=active, and Result=success confirms nothing failed on the last run. ConditionResult=no alongside ActiveState=inactive means the unit was skipped, and the journal line naming the condition tells you which test failed.

FAQ

Does Requires= wait for the other unit to start?

No. Requires= and After= are separate settings. Requires= pulls the other unit into the same transaction, then systemd starts both jobs in parallel. To wait, add After= naming the same unit. There is a second reason to add it: a Requires= dependency that fails only prevents your unit from starting when After= is also set, because without ordering your unit has already been started by the time the other one fails.

Should I order after network.target or network-online.target?

At boot, network.target only means the network management software was started, so it promises nothing about addresses or routes. Use network-online.target when your service needs a working address at start, and write both Wants=network-online.target and After=network-online.target, because the target is not in the default boot transaction and After= alone waits for a unit nobody queued. If the service fails only because it binds one specific IP, net.ipv4.ip_nonlocal_bind=1 with Restart=on-failure is lighter than delaying boot.

Why does my unit report success but never run?

A failed Condition...= test skips the unit and reports the start job as successful, so nothing is ever marked failed. Run systemctl show <unit> -p ConditionResult, and ConditionResult=no confirms it. Then read journalctl -b -u <unit> for the line Condition check resulted in <description> being skipped. On systemd 250 and newer, systemctl status <unit> also names the exact directive that was not met.

What is the difference between Condition and Assert?

They run identical tests. A failed Condition skips the unit quietly and the job still succeeds. A failed Assert fails the unit, logs Assertion failed for <description>. and leaves it in failed (Result: assert). Use Condition for "this unit does not apply on this machine", which covers almost every real case. Use Assert only when a missing precondition has to be visible to whoever watches failed units.

Why does ExecStartPre fail with status=203/EXEC?

203/EXEC means systemd could not execute the command at all. The usual causes are a path that is not absolute, a binary that does not exist on that machine, a file without the execute bit, or a script whose #! line points at a missing interpreter. systemd's other small codes come from a fixed table, so status=2/INVALIDARGUMENT just means the command exited 2 and says nothing about arguments. Remember that ExecStartPre= is not run through a shell, so pipes and globs need /bin/sh -c '...'.