Why cron job no dey run and how to fix am

Why your cron job no dey run

Cron job wey “never run” almost always don run. E run for environment wey no be your shell, e fail for the first second, and the message go somewhere wey you no dey read. Five causes explain almost every report: search path, percent sign, wrong crontab file, output wey go mail, and script wey expect login session.

cron na daemon (background service) wey dey read crontab files and start commands according to schedule. E no dey read your .bashrc, e no dey open terminal, e no dey start login shell, and e no dey tell you when command fail. Every cause below come from these four facts.

Go through dem in order, and start with the question wey dey under all of dem: cron fire at all? “cron never start the job” and “the job start and die” na different problems wey get nothing in common, so answer that one first.

Cron run at all?

The daemon fit get different unit name for different distribution family. Check both, then read the log.

systemctl status cron
systemctl status crond
journalctl -u cron --since "2 hours ago"
journalctl -u crond --since "2 hours ago"

Debian and Ubuntu dey call the unit cron. Fedora, Rocky and Alma dey call am crond. Na only one of these names dey exist for one machine, so if one of the two commands report unknown unit, e normal and e no be fault.

Read the entries wey your own system write. No go find line wey guide copy, because the wording dey differ between cron implementations and different logging setups. Na two things you dey check only: whether entry dey for the minute wey your schedule specify, and whether that entry name your command. Entry wey name your command mean say cron don do im part, and the failure dey inside the command. If no entry dey at all, cron never get your schedule, and that one na cause 3 below.

Some images dey send cron messages through rsyslog enter file instead of journal. Check /var/log for file wey cron or syslog name, then read the end of the file.

ls -l /var/log
sudo tail -n 50 /var/log/syslog

If neither the unit nor the log dey exist, cron fit simply no dey installed. Minimal cloud images and containers often no include am.

dpkg -l cron
rpm -q cronie
sudo apt install cron
sudo dnf install cronie
sudo systemctl enable --now cron

Cause 1: cron no get your PATH

Your interactive shell dey build PATH from /etc/profile, ~/.profile, ~/.bashrc and everything wey those files source. None of dem dey run for cron job. cron dey start the command with its own short environment, so e no go find program wey dey outside standard system directories. Anything under /usr/local/bin, /opt, language version manager, Python virtual environment or Go workspace fit cause am. The job go fail for its first line, and the shell go write "not found" style error. The exact wording dey depend on the shell wey run am.

Find the real path of every command wey your job dey use.

command -v docker
command -v node
readlink -f "$(command -v node)"

Then either write those absolute paths inside the job, or set PATH once for the top of the crontab.

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
0 3 * * * /usr/local/bin/mytool run

Use your own machine run echo "$PATH" to collect that list, then remove anything wey only dey exist inside interactive session. One rule matter here: cron no dey expand variables for these assignment lines. PATH=$PATH:/usr/local/bin go store the literal text $PATH:/usr/local/bin, so the job go end up with search path wey no get any usable directory. Write the complete list out.

Version manager need more than path. nvm, pyenv, rbenv and asdf dey install shell function or shims directory from your .bashrc, and cron job never dey read that file. Call the versioned binary with absolute path, or source the manager's init script as the first line of your own script.

Cause 2: percent sign dey end your command

For the command field of a crontab, % no be ordinary character. The first percent sign wey no get escape, %, dey end the command. Everything after am, cron dey pass to the command as standard input, and every further % dey turn to newline. Na real cron feature be this for feeding short input give program, and na also why date-stamped filename na the classic broken crontab entry.

Write 0 3 * * * /usr/bin/tar -czf /srv/backups/site-$(date +%F).tar.gz /srv/site and tar no go ever see formatted date. cron go cut the line for the first %, so shell go receive unfinished command substitution, while the rest of your line go arrive as standard input. Escape every percent sign with backslash.

0 3 * * * /usr/bin/tar -czf /srv/backups/site-$(date +\%F).tar.gz /srv/site

Two layers dey read that one line, one after another. \% na cron rule, and cron dey apply am before e start anything. $(date +\%F) na command substitution, and shell wey cron start dey apply am later. To know which layer own each character na the main trick.

The safer habit na to keep logic comot from crontab completely. Put am inside script, where percent sign no get special meaning.

#!/bin/bash
set -euo pipefail
stamp="$(date +%F)"
tar -czf "/srv/backups/site-${stamp}.tar.gz" /srv/site

The crontab line go then contain path and redirect only, nothing else. Crontab wey you fit read with one quick look na crontab wey you fit debug.

Cause 3: na which crontab you edit?

No be one crontab dey. Several files dey, with different owners and different number of fields. If you write job for wrong one, you no go see am.

• crontab -e dey edit crontab of the user wey run the command. sudo crontab -e dey edit root own. Two people wey dey debug the same box fit end up dey read two different files.
• sudo crontab -l -u deploy dey list another user's crontab. Na so you confirm wetin actually install for the account wey suppose run the job.
• /etc/crontab and every file for /etc/cron.d get one extra field between the schedule and the command: the user wey go run am as. If you paste five-field user crontab line inside /etc/cron.d, the first word of your command go read as username.
• Files for /etc/cron.d must get names wey use letters, digits, underscores and hyphens only. File wey dem call backup.sh or site.conf go skip because of the name alone. Rename am to backup and check your log again.
• Files for /etc/cron.d suppose belong to root, and group or other users must not get write permission. ls -l /etc/cron.d show you both facts at once.
• Scripts wey you drop inside /etc/cron.daily and the other similar directories follow the same naming rule, and dem must also get execute bit. If execute bit dey miss, cron go skip am silently.
• /etc/cron.allow and /etc/cron.deny decide who fit install crontab at all. If either one dey for your box, read am before you assume say your user fit get one.

Install user crontab with crontab command instead of editing spool file by hand, because crontab go parse the file before e install am. When you save, read wetin the command print back to you. If e reject the file, the previous version go remain active and your change no go take effect. This look exactly like cron dey ignore you.

The owner also decide permissions. Job for root's crontab dey create root-owned files wey application wey dey read dem fit no get permission to write. Job for normal user's crontab no fit read root-only directory. Match the owner with the work: application maintenance suppose belong to the application's own account. Na this reasoning dey behind replacing WordPress wp-cron with a system cron job. The mode of files wey your job create come from the umask wey e inherit. That value no be your shell's umask, so how umask sets file permissions worth reading if job output dey land unreadable.

Cause 4: output go mail wey nobody dey read

cron dey collect everything wey job write to standard output and standard error. If job write anything at all, cron dey hand that text go local mail system, address am to crontab owner or to anything wey MAILTO name. For stripped-down VPS, most times no MTA (mail transfer agent) dey installed, so nothing deliver am. Your error dey exist for small time, then e disappear without going anywhere. Na this make broken job look silent.

Send the output go file wey you control instead.

0 3 * * * /usr/local/sbin/backup-site.sh >> /var/log/backup-site.log 2>&1

>> dey append standard output to the file. 2>&1 dey point standard error to wherever standard output dey point at that time, so e must come after the redirect. If you write am the other way, as 2>&1 >> file, standard error go keep the original destination, and the error wey you dey find na exactly the part wey no reach the file.

The journal na another good target. logger dey write into syslog under tag wey you choose.

0 3 * * * /usr/local/sbin/backup-site.sh 2>&1 | logger -t backup-site

Read am back with journalctl -t backup-site. This one keep the job output near the cron entries, so e easy to follow the timeline. If you also need record of which person run which command for the box, na separate system be that, and auditing user commands for your server cover am.

MAILTO="" for the top of crontab dey turn mail off for the jobs below am. Setting MAILTO to real address only help if working MTA dey, so first prove say mail dey leave the box before you depend on am.

One rule as you dey debug: never append > /dev/null 2>&1. Na the most common line for every crontab, and e dey throw away the only evidence wey you get. Put am back later if you want, after the job don work.

Cause 5: script dey assume environment wey cron no dey provide

Once dem find the command and capture the output, everything wey remain na wetin your session dey give you free.

• Shell fit no be bash. Check am with ls -l /bin/sh. For Debian and Ubuntu e dey point to dash, so double bracket test, arrays and source go fail with syntax error. Give script #!/bin/bash line and call the script, or set SHELL for top of crontab.
• Working directory no be the place wey you dey before. Use absolute paths everywhere, or cd go the directory for first line of script. Relative path na the commonest single reason why job "dey work when I run am by hand".
• Locale no be the one for your session. Anything wey format date or number, or sort text, fit produce different output under different LANG. If later step parse that output, set locale for script instead of hoping.
• No TTY (terminal) dey. Command wey ask for confirmation, open editor or draw progress bar fit hang or exit. Add any non-interactive flag wey the tool get.
• No SSH agent dey. SSH_AUTH_SOCK no dey for cron environment, so ssh or rsync command wey work because your agent dey loaded now go fail to authenticate. Give the job its own key, and make the job user own am.
• No user session bus dey, so systemctl --user from cron job go fail until XDG_RUNTIME_DIR set. System unit na better answer.

For Fedora, Rocky and Alma, one more suspect dey. SELinux dey confine cron jobs, so job wey touch path with unexpected label go get denied even when file permissions look correct. Check denials with sudo ausearch -m avc -ts recent, and read SELinux basics for server before you switch anything off.

The one minute probe wey go show you cron environment

Stop dey guess wetin cron environment dey carry; read am directly. Write script wey go dump everything, schedule am every minute, wait, then read the file.

cat > /home/deploy/cron-probe.sh <<'EOF'
#!/bin/bash
echo "=== probe ==="
date -Is
pwd
id
echo "SHELL=$SHELL"
echo "LANG=$LANG"
command -v node || echo "node is not on this PATH"
env | sort
EOF
chmod +x /home/deploy/cron-probe.sh

Add one line to the crontab of the user wey the real job dey run as. Use absolute paths for both sides.

* * * * * /home/deploy/cron-probe.sh >> /home/deploy/cron-probe.log 2>&1

Wait one minute, then read /home/deploy/cron-probe.log and compare am with the same commands wey you run for your own shell. The PATH line, working directory, and locale usually dey explain the failure by themselves. Note 2 details for the setup: percent signs dey inside the script, where cron rule no apply, and the log path na path wey the job user fit write to.

Delete that crontab line immediately after you get your answer. Job wey dey run every minute and append to file fit fill small disk, and e go do am quietly.

Schedule na the one you mean?

A user crontab line dey start with five fields: minute, hour, day of month, month, day of week. Two of dem dey interact for way wey fit surprise people.

When day of month and day of week both get restriction, meaning say neither one be *, cron go run the job when either field match. 0 0 13 * 5 no mean "Friday the 13th". E go run for midnight on the 13th of every month, and for midnight every Friday. To get one specific day, leave one of the two fields as * and test the other one inside the script.

cron dey use system timezone. Many VPS images dey configured to UTC (coordinated universal time), so job wey you schedule for 03:00 go run for 03:00 UTC, and that time fit be middle of your afternoon. timedatectl go print the timezone wey your box actually dey use. Read your own value instead of assuming say e match your laptop.

Two more schedule traps dey worth knowing. @reboot go fire when cron itself start, and that no be the same time network don ready. So job wey need DNS or remote host fit fail during boot and succeed for every manual run after that. Also, nothing stop slow job from starting again while the previous copy still dey run. Put am inside a lock.

*/5 * * * * /usr/bin/flock -n /tmp/backup-site.lock /usr/local/sbin/backup-site.sh >> /var/log/backup-site.log 2>&1

flock -n go stop immediately when another process already hold the lock, so the overlapping run go stop instead of piling up on top the first one.

When systemd timer na the better tool

cron good for one thing: run this command for this time. E weak for almost everything else. Timer dey give you journal without any redirect, exit status wey you fit query later, ordering against network-online.target, and randomized delay so hundred servers no go all start for the same second. When your job need any of these things, systemd service and timer for VPS na less work than defending one crontab line. Retry behaviour belong there too, because systemd restart policies decide wetin go happen after failure, and cron no get any answer for that question.

Keep cron for small jobs. Move anything wey get dependencies or retry policy go timer. Both fit run for the same server, so you no need finish this migration for one sitting.

FAQ

Cron job by hand dey work but e fail from cron, why?

Na because your shell environment and cron environment no be the same. Your login shell dey read /etc/profile and ~/.bashrc, wey dey set PATH, the locale, and your agent variables. cron dey start the command without any of those settings, from another working directory, and sometimes with another shell. Use absolute paths for every command. Set everything wey you need for the top of the crontab or inside the script. Schedule one probe job for one minute later wey go run env | sort, pwd, and id into a log file. This one go let you read cron real environment instead of guessing.

How I fit check whether cron really run my job?

Read the daemon log. Use journalctl -u cron for Debian and Ubuntu, or journalctl -u crond for Fedora, Rocky, and Alma. Some images dey route the messages through rsyslog into a file under /var/log instead. Look for an entry for the minute wey your schedule specify, and confirm say e name your command. If no entry dey, cron never get the schedule. Confirm say you edit the correct crontab. If entry dey but no result dey, the command start and die. Capture the output with a redirect.

Why date +%Y dey break inside crontab?

cron dey treat % as special for the command field. The first unescaped % dey end the command. Everything after am dey pass to that command as standard input, and every further % becomes a newline. So filename wey use date format no go reach the program wey you write am for. Escape every percent sign as \%, or move the command into a script and call the script from cron. Inside a script, the percent sign no get special meaning.

Where cron job output dey go?

E dey go the local mail system, addressed to the crontab owner or to whatever MAILTO name. Most VPS images no get mail transfer agent installed, so the message dey discard and the job look silent. Redirect the output to a file with >> /path/to/log 2>&1. Keep that order so standard error follow standard output. Or pipe am through logger -t myjob and read am back with journalctl -t myjob. No use > /dev/null 2>&1 while you still dey debug.

I suppose use cron or systemd timer?

Use cron for simple command wey suppose run for fixed time, especially if you fit need move am to machine wey no dey run systemd. Use a timer when you want the output for the journal without redirect, an exit status wey you fit query, ordering after the network don come up, randomized start delay, or retry policy after failure. Both fit run for the same server, so you fit move the jobs one by one as dem become suitable.