Why systemd No Restart Your Service When Child Die

Short answer: systemd restart policies dey monitor one process

systemd restart policies dey monitor one process for each unit: the main process. Restart= dey read the exit status of only that process. A unit control group fit hold twenty processes. One of dem fit die, but the unit go remain active (running) because the main process still dey run. As far as systemd concern, nothing fail, so e no go restart anything.

systemd dey know about the other processes. E dey kill dem when the unit stop, e dey count their memory against the unit limits, e dey apply the unit CPU quota to dem, and e dey show dem for systemctl status. But e no dey ever read their exit status. The restart logic and the cgroup na two different things, and most of this guide dey explain the gap between dem.

Wetin cgroup dey hold, and wetin restart logic dey read

A cgroup (control group) na kernel object wey dey own set of processes. Every service unit get one, and dem name am after the unit. Process no fit comot from am. Children dey inherit the cgroup of their parent, and unprivileged process no fit move itself go another place. Na why systemd fit clean up daemon wey fork two times, something wey old init scripts no fit do reliably.

Look both facts side by side:

systemd-cgls --unit myapp.service
systemctl show -p MainPID -p NRestarts -p Restart -p RestartUSec myapp.service

systemd-cgls list every process for the unit. MainPID na the only number wey restart policy dey read. When both no agree with how you understand the system, na that disagreement be the bug. MainPID=0 worse pass wrong PID: e mean say systemd no dey track anything at all, so no Restart= value fit ever fire.

One real exception dey to the main-process rule. If kernel out-of-memory killer kill any process inside the unit's cgroup, systemd go see am, because e dey watch the cgroup's memory.events file. OOMPolicy= decide wetin go happen next, and the default na stop: the whole unit go stop, the result go record as oom-kill, and that one count as failure, so Restart=on-failure go fire. The journal talk am plainly.

myapp.service: A process of this unit has been killed by the OOM killer.
myapp.service: Failed with result 'oom-kill'.

So, if memory kill child, e go bring the unit down, but if the same child die from segmentation fault, e no go. If you set memory limits for a unit, read how MemoryMax and CPUQuota dey apply to a unit's cgroup before you tune restart policy, because na here these two features meet, and nowhere else.

How Type= dey choose the main process

Type= for [Service] section no be only about start-up ordering. Na the rule wey decide which PID (process ID) go become MainPID, and na the same thing as deciding wetin Restart= fit see.

• Type=simple na the default. The process wey systemd fork from ExecStart= na the main process. systemd mark the unit as started immediately, before e know whether exec even work. Typo for binary path go give you start job wey succeed, then Main process exited, code=exited, status=203/EXEC one moment later.
• Type=exec behave like simple, but the start job go wait until exec don succeed. This turn the typo wey dey above into correct start failure. E need systemd 240 or newer, and every supported distribution get am. Prefer am instead of simple.
• Type=forking expect the process from ExecStart= to fork background daemon, then exit. systemd go wait for the parent to exit, then look for the real daemon. Give am PIDFile=. Without one, GuessMainPID= (on by default) go work only when exactly one process remain for the cgroup. If two remain, MainPID go stay 0.
• Type=notify mean say the service go call sd_notify(3) and send READY=1 when e fit serve traffic. E fit also send MAINPID= to give systemd another process to track. NotifyAccess= dey default to main, so systemd go ignore notification wey child send, and the journal go name the PID wey send am.
• Type=oneshot no get main process wey remain. The unit go become inactive as soon as ExecStart= finish, unless you set RemainAfterExit=yes. Restart=always and Restart=on-success no dey allowed here, with the message Service has Restart= set to either always or on-success, which isn't allowed for Type=oneshot services. Refusing.. Other values, including on-failure, dey accepted.

Two Type=forking errors good make you memorise dem, because each one fit leave you with unit wey look broken without any visible reason:

myapp.service: Can't open PID file /run/myapp.pid (yet?) after start: No such file or directory
myapp.service: New main PID 4711 does not belong to service, and PID file is not owned by root. Refusing.

The first one mean say the daemon write its PID file for another place, or write am later than when systemd check. The second one mean say the PID file name process wey dey outside the unit's cgroup. systemd refuse to adopt am because writable PID file fit otherwise become way to make systemd send signals to any process for the machine.

Why wrapper script dey hide when e children die

Na this setup dey cause the question for the heading.

#!/bin/bash
/usr/local/bin/myapp-web &
/usr/local/bin/myapp-worker &
wait

The unit na Type=simple, so shell na the main process. wait with no arguments only returns after every child don exit. Kill the worker and shell go continue to wait for the web process. So shell no go exit, MainPID no go exit, and Restart= no go ever check. The cgroup don get one process less, systemctl status go print the shorter tree, and the unit still dey active (running). Nothing for systemd dey watch that tree for changes.

Another version of the same mistake dey happen quietly:

ExecStart=/bin/sh -c 'export APP_ENV=production; /usr/local/bin/myapp'

Shell na the main process, no be myapp. For systemctl stop, systemd sends SIGTERM go the main process, but shell wey dey wait for foreground child no dey pass the signal go am. The stop go take the full TimeoutStopSec, wey be 90 seconds by default, and e go end like this:

myapp.service: State 'stop-sigterm' timed out. Killing.
myapp.service: Killing process 4711 (myapp) with signal SIGKILL.

The fix na exec. Write exec /usr/local/bin/myapp and shell go replace itself with the program, so MainPID go be the program and signals go reach am. Better still, remove shell and use Environment= or EnvironmentFile= for the unit. Note say this bug dey hide itself when the -c string get only one command, because bash and dash both optimise that case into direct exec. Add a second command to the string and shell go remain alive in front of your program.

The full Restart= vocabulary, and when on-failure beats always

Restart= get seven values, and the main difference between dem na wetin count as clean. systemd dey treat exit code 0, any code wey dey inside SuccessExitStatus=, plus signals SIGHUP, SIGINT, SIGTERM and SIGPIPE as clean exit. Everything else, including SIGKILL and SIGSEGV, no clean.

• no na the default. The unit no go restart by itself. Na why unit wey no get Restart= line go stop after the first crash and remain stopped.
• on-success go restart only after clean exit.
• on-failure go restart after non-zero exit code, unclean signal, start or stop timeout, or watchdog expiry.
• on-abnormal go restart after unclean signal, timeout or watchdog expiry, but e no go restart after ordinary non-zero exit code.
• on-abort go restart only after unclean signal, wey mean crash.
• on-watchdog go restart only when WatchdogSec= expire.
• always go restart after every case wey dey above, including clean exit with status 0.

on-failure na the correct default for long-running daemon. E go bring service back after crash, but e go leave deliberate exit 0 alone. always fit work for program wey dey exit cleanly because of something outside its control, like tunnel client wey dey return 0 when the far end disconnect. The problem with always be say e fit hide bugs: service fit start, read broken config file, log the error and exit 0, then e go loop forever. The only sign fit be restart counter wey dey increase.

SuccessExitStatus= dey change the boundary between clean and unclean. Borg dey exit 1 for warnings and 2 for errors, so backup unit wey no get SuccessExitStatus=1 go dey marked as failed every time e skip one unreadable file. RestartPreventExitStatus= dey list codes wey go block restart even under always. Na the clean way for program to talk say e no suppose come back. RestartForceExitStatus= dey do the opposite. Backup job suppose dey inside Type=oneshot unit wey timer dey drive, instead of restart loop. the service and timer pair wey dey run job on schedule na the pattern to copy for there.

One warning about testing. If you kill your service with plain kill <pid>, e go send SIGTERM. SIGTERM dey the clean list, so Restart=on-failure correctly no go do anything, and you fit conclude say your config spoil. Use kill -9 <pid> or systemctl kill -s SIGKILL myapp.service instead. Also remember say no value of Restart= go fire after systemctl stop, or when unit stop because BindsTo= or PartOf= dependency don disappear. Stop job no be failure.

RestartSec, and the 100 millisecond default

RestartSec= na the pause between when the unit stop and when systemd start am again, and the default na 100 milliseconds. Check wetin your unit actually load:

systemctl show -p RestartUSec -p StartLimitIntervalUSec -p StartLimitBurst myapp.service

Unit wey never set am go print RestartUSec=100ms. That default dey okay for service wey crash once and come back. E no correct for service wey no fit start at all, because five restarts go happen inside half a second. Na this exactly dey trigger the rate limit wey we go describe next. For anything wey dey wait for database, mount, or network route, set RestartSec=5s or higher.

As of August 2026, systemd 254 and newer also get RestartSteps= and RestartMaxDelaySec=. Dem dey increase the delay from RestartSec= reach a maximum over that number of attempts. Ubuntu 24.04 dey ship systemd 255 and get dem. Debian 12 dey ship systemd 252 and no get dem. Growing delays na the correct solution when dependency fit stay down for long time.

Wetin “start request repeated too quickly” really mean

Na this state make readers think say systemd dey give up anyhow. E be counter. The rule be say: if dem start one unit pass StartLimitBurst= times inside StartLimitIntervalSec=, systemd go refuse to start am again and put am for failed state. The default na 5 starts inside 10 seconds.

The journal go show the sequence:

myapp.service: Scheduled restart job, restart counter is at 5.
myapp.service: Start request repeated too quickly.
myapp.service: Failed with result 'start-limit-hit'.
Failed to start myapp.service - My application.

and systemctl start go answer with the fix wey don already write out:

Job for myapp.service failed because start of the service was attempted too often. See "systemctl status myapp.service" and "journalctl -xeu myapp.service" for details. To force a start use "systemctl reset-failed myapp.service" followed by "systemctl start myapp.service" again.

systemctl reset-failed myapp.service go clear the counter and the failed state. Nothing else fit do am, so plain systemctl start go continue to face refusal until you run am. Manual starts still count toward the limit, so some impatient systemctl restart runs while you dey edit config file fit trigger am even when no crash happen.

The part wey dey mislead people be say: start-limit-hit no dey ever talk why the service dey fail. E only talk say e fail repeatedly and quickly. The real reason dey for the journal lines above am.

Both settings belong inside the [Unit] section. You go see examples wey put dem for [Service], wey older systemd accept, and na there the confusion start. Write dem for [Unit], then ask systemd wetin e load with systemctl show, because na the loaded value dey count.

[Unit]
Description=My application
StartLimitIntervalSec=300
StartLimitBurst=5

[Service]
Type=exec
ExecStart=/usr/local/bin/myapp
Restart=on-failure
RestartSec=10s

That one go give the unit five attempts inside five minutes before e give up. StartLimitIntervalSec=0 go turn off the limit completely, and you need understand wetin you choose: service wey no fit ever start go now retry forever and write to the journal every time. The machine-wide defaults dey for /etc/systemd/system.conf as DefaultStartLimitIntervalSec= and DefaultStartLimitBurst=.

One nearby setting need warning. StartLimitAction= decide wetin go happen when the limit reach, and e accept values including reboot, reboot-force and poweroff. The default na none, wey fail the unit and leave the machine as e be. For remote VPS, poweroff mean say the box go remain off until you open the provider’s console.

Fix one: one process per unit

For almost every case, na be this answer. If two programs must run, write two units. Each unit go then get one real main process, one real exit status, and its own restart policy. You go also get separate logs, separate resource limits, and separate restart counters, wey na wetin you want for 3 a.m.

Show the relationship between the units inside the unit files, no be inside shell script.

• After= only control start-up order. E no talk anything about failures.
• Requires= start the other unit together with this one, and e stop this one if person stop the other one explicitly.
• BindsTo= na Requires= plus the case wey matter here: this unit go stop when the other one stop for any reason, including crash. Pair am with After=, or the ordering no get defined result.
• PartOf= pass stop and restart downwards, so systemctl restart myapp.target go reach every unit wey be PartOf= am.
• Upholds= (systemd 249 and newer, so Ubuntu 22.04 and later) keep the named unit running: if e stop, systemd go start am again. E still dey subject to the same start rate limit like everything else.

One worker wey must never run without its API server, and wey systemd go keep alive anytime the API dey up:

# /etc/systemd/system/myapp-api.service
[Unit]
Description=myapp API server
Wants=network-online.target
After=network-online.target
Upholds=myapp-worker.service

[Service]
Type=exec
User=myapp
ExecStart=/usr/local/bin/myapp serve
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/myapp-worker.service
[Unit]
Description=myapp background worker
BindsTo=myapp-api.service
After=myapp-api.service
StartLimitIntervalSec=120
StartLimitBurst=5

[Service]
Type=exec
User=myapp
ExecStart=/usr/local/bin/myapp worker
Restart=on-failure
RestartSec=5s

The worker no get [Install] section, and nobody enable am by hand. The API unit pull am in with Upholds=, so systemctl enable --now myapp-api.service na the only command wey you run. Reload and check wetin systemd create for the two units:

sudo systemctl daemon-reload
systemd-analyze verify /etc/systemd/system/myapp-worker.service
systemctl list-dependencies myapp-api.service

systemd-analyze verify no print anything at all when the file clean. Any output mean say problem dey. Usually na key wey systemd no recognise for the section where you write am, or dependency on unit wey no exist.

Fix two: Type=notify, so systemd fit know pass PID

If program dey speak systemd notification protocol, use am. With Type=notify service go tell systemd when e ready. This make ordering real instead of guesswork. E fit also send MAINPID= to show systemd the process wey matter, instead of launcher.

WatchdogSec= na the part wey worth the effort. Set am, and service must send WATCHDOG=1 through sd_notify(3) at least that often. When messages stop, systemd go terminate service with SIGABRT and mark am failed. So Restart=on-failure or Restart=on-watchdog go bring am back. Na this one be the only built-in way to restart process wey still dey alive but don hang. No exit-status policy fit catch that.

[Service]
Type=notify
NotifyAccess=main
ExecStart=/usr/local/bin/myapp serve
WatchdogSec=30s
Restart=on-failure
RestartSec=5s

Watchdog trip go show for journal as myapp.service: Watchdog timeout (limit 30s)!, followed by the kill. But if unit sit for activating (start) until TimeoutStartSec finish, READY=1 never arrive. Either program no dey speak the protocol, or NotifyAccess=main dey reject notification wey come from child process. Journal go report both PIDs.

For software wey get HTTP health endpoint but no sd_notify support, the honest options na small timer unit wey go probe endpoint and call systemctl restart, or make container runtime do the probing. Na for this reason Compose healthchecks and how dem restart dey exist.

Fix three: supervisor wey dey inside unit, only when choice no dey

Some software really dey ship as bundle of processes behind launcher wey you no fit split apart. For this case, you run supervisor inside unit and accept the result: systemd dey watch supervisor, supervisor dey watch everything else, and your restart policy now dey inside two files.

The common example na container runtime. A docker compose or podman unit follow exactly this pattern. The per-container restart policy dey inside Compose file, while the systemd unit only keeps the runtime running. If na your setup be this, the unit wey bring Compose stack up during boot show the working version, including why Type=oneshot with RemainAfterExit=yes usually correct for there.

The cgroup still dey work for your benefit. Everything wey supervisor start remain inside the unit's cgroup. So MemoryMax=, CPUQuota=, and the cleanup when service stop still cover the whole process tree. Na only the restart decision supervisor dey handle.

Any supervisor wey you choose, no set Restart=always for outer unit and aggressive restart policy inside am without proper consideration. Two layers of restart logic, each with its own backoff, fit make service dey flap for minutes. The journal fit also fail to explain why.

ExitType=cgroup no mean “restart when any process die”

ExitType= (systemd 250 and newer, so Ubuntu 24.04 and Debian 12 both get am) na the setting people dey find when dem search for this problem, and e dey do the opposite of wetin the name suggest. The default, ExitType=main, mean say system consider the service stopped when the main process exit. ExitType=cgroup mean say system consider the service still dey run until the last process for the cgroup exit.

So ExitType=cgroup make unit less sensitive to one process wey die, no be more sensitive. Na the correct setting for program wey fork its real worker, then parent exit without writing PID file, where Type=forking no fit find the daemon. But na wrong setting for the failure wey we describe here.

No Restart= value dey wey mean “restart the unit when any process for the cgroup die”. If you need that behaviour, you need one process for each unit. If you no fit split the program and na you control the wrapper script, the nearest option na wait -n, wey return as soon as the first child exit:

#!/bin/bash
/usr/local/bin/myapp-web &
/usr/local/bin/myapp-worker &
wait -n
exit 1

Any child wey die go now bring the wrapper down with non-zero status, so Restart=on-failure go act. This na compromise, no be fix. You still get one restart counter for two programs, one log stream, and no way to restart the failing half by itself.

Wetin actually happen, how to inspect am

Four commands, run am for this order.

systemctl status myapp.service
systemd-cgls --unit myapp.service
systemctl show -p MainPID -p NRestarts -p Result -p ExecMainStatus myapp.service
journalctl -u myapp.service -b -o short-precise

systemctl status dey show the state, main PID, and cgroup tree for one screen. Healthy unit go show Active: active (running) with one Main PID: line wey name the process wey you expect. If the tree for bottom list processes wey you no recognise, or e no include one wey you recognise, you don already get your answer.

systemd-cgls --unit dey print the same tree without truncation. This one matter once unit dey hold more than small number of processes.

systemctl show dey give machine-readable facts. NRestarts= na the restart counter, and na the fastest way to know service wey don restart forty times from one wey don dey up since boot. Result= hold the last failure reason: exit-code, signal, timeout, oom-kill, watchdog or start-limit-hit. ExecMainStatus= na the raw exit status of the last main process.

Journal hold the sequence of events. Na these three lines you need search for:

myapp.service: Main process exited, code=exited, status=1/FAILURE
myapp.service: Failed with result 'exit-code'.
myapp.service: Scheduled restart job, restart counter is at 1.

code=exited, status=N mean say program choose to return N, so the fault dey inside the program or its configuration. code=killed, signal=SEGV mean say e crash. code=killed, signal=TERM usually mean say another thing tell am make e stop. This one no be failure and e no go trigger Restart=on-failure. code=dumped mean say e leave core file, wey coredumpctl list go show you when systemd-coredump dey installed.

For more than one machine, NRestarts na the number wey make sense to collect on schedule. Unit wey counter dey increase every day dey fail every day, whether anybody notice am or not. Once you don pass two or three boxes, consistent way to run one command for every server go change this from guesswork to report.

FAQ

Why systemctl dey talk say my service active when process don die?

systemd dey track one process for each service unit, na the main process, and Restart= dey read only the exit status of that process. Everything wey the unit start dey inside the same cgroup. systemd go kill those processes when the unit stop, but e no dey monitor dem for exit. Run systemctl show -p MainPID myapp.service and compare the number with systemd-cgls --unit myapp.service. If the process wey die dey show for the tree but e no be MainPID, systemd behave exactly as dem design am. The fix na make one process dey for each unit, and write the relationship as BindsTo= and Upholds= between the units.

Wetin "start request repeated too quickly" mean?

E mean say dem start the unit more than StartLimitBurst= times inside StartLimitIntervalSec=. The default na 5 starts for 10 seconds, so systemd stop to try again. Na rate limit be this, and e no dey explain why the service fail, so read the journal lines wey dey above am. Clear the state with systemctl reset-failed myapp.service, then fix the underlying failure. If the service dey wait for something slow to start, increase RestartSec=, because the default gap of 100 milliseconds go use all five attempts in less than one second.

I suppose use Restart=always or Restart=on-failure?

Use on-failure for almost everything. E go restart after crash, non-zero exit, timeout, or watchdog trip, but e go leave deliberate exit 0 alone. Use always only when the program exit cleanly for reasons wey no dey under its control, like client wey return 0 when its peer disconnect. The problem with always na say service wey read broken config, log one error, then exit 0 fit loop forever. The only visible sign na NRestarts dey increase for systemctl show.

Why killing my process by hand no dey trigger restart?

Because systemd dey count SIGHUP, SIGINT, SIGTERM and SIGPIPE as clean exits, and plain kill <pid> dey send SIGTERM. Under Restart=on-failure, clean exit no be failure, so nothing go restart and the configuration go look broken even though e no be. Test with kill -9 <pid> or systemctl kill -s SIGKILL myapp.service. Dem be unclean termination, and dem go trigger the policy. The same rule explain why systemctl stop no dey fight your restart policy.

Where StartLimitIntervalSec and StartLimitBurst dey go?

Dem dey inside the [Unit] section. Older material and older systemd versions put dem for [Service], so copied examples no agree with each other. No guess which one your version dey honour. After systemctl daemon-reload, ask systemd wetin e load with systemctl show -p StartLimitBurst -p StartLimitIntervalUSec myapp.service, and treat those numbers as the correct values. systemd-analyze verify /etc/systemd/system/myapp.service dey catch keys wey systemd no recognise at all, and e print nothing when the file clean.