Ubuntu upgrade failed halfway: how to recover

Ubuntu upgrade failed halfway: read the symptom first

A failed Ubuntu release upgrade from 24.04 to 26.04 leaves the server in one of four states, and each state has a different fix. The upgrader may still be running inside a screen session you lost contact with. dpkg may have been killed with a package half configured, so apt now refuses every command. The apt sources may name 26.04 while the installed packages are still 24.04. Or the server may not boot at all. Work out which of those you have before you type anything, because the fix for one state can make another worse.

One rule covers all four. Do not reboot until you know what state dpkg is in. A reboot in the middle of a package swap is what turns a fixable dpkg interruption into the will-not-boot case near the end of this guide. And do not start a second apt or dpkg process while the first one might still be alive, because two writers on the package database is how the database gets corrupted.

This guide assumes you followed the 24.04 to 26.04 upgrade guide and took a snapshot before you started. If you did not, read the boot section with that in mind, since the snapshot is the answer to the worst case.

Is the upgrade still running?

Many upgrades reported as failed are still running. The SSH session dropped, the terminal went blank, and the upgrader carried on without you.

do-release-upgrade is built for this. When it runs with its text interface, which is what a server gets, it wraps itself in a GNU screen session, so the upgrade survives the loss of the terminal that started it. Separately, when it detects that it was started from an SSH session, it offers to start a second sshd on another port (1022 by default) so you can still log in if the main SSH daemon breaks during the package swap. Both facts matter now.

Reconnect over SSH and look for the screen session. The upgrader ran under sudo, so its screen session belongs to root, and a plain screen -ls as your own user will not list it.

sudo screen -ls

screen -ls marks each session as attached or detached. If one is listed, reattach to it. If it is still marked attached because the dead SSH connection never released it, -d detaches that stale attachment first.

sudo screen -d -r

If more than one session is listed, put the session name from screen -ls after -r. Running sudo do-release-upgrade again also works: the upgrader checks for its own existing screen session and reattaches instead of starting a new run. Either way you land back in the running upgrade, which is usually waiting at a prompt about a changed config file or a service restart. Answer it and let it finish.

If you started the upgrade inside tmux, as the upgrade guide recommends, reattach tmux first with tmux attach. The screen session is running inside that tmux pane, so you see the upgrade directly. If the pane holds only a shell prompt, the upgrader is no longer running there, and sudo screen -ls is the next check.

If the main SSH port refuses the connection, try the fallback port: ssh -p 1022 user@host. That daemon exists only for the life of the upgrade, so if you cannot get in on either port, use your provider's console instead. From the console, sudo ss -ltnp shows which ports an sshd process is listening on, and sudo ufw status tells you whether the firewall let the fallback port through.

When no screen session exists and nothing is waiting on the console, the upgrade really did stop. Confirm nothing is still working on the package database before you touch it:

ps -eo pid,etime,cmd | grep -iE '[u]pgrade|[d]pkg|[a]pt'

An empty result means dpkg is idle and you can move on to repairing it. A dpkg or apt process with a long elapsed time and no screen session to reattach is hung. Give it a few minutes, check the console for a debconf question nobody answered, and only then kill it. Never delete the lock files under /var/lib/dpkg/ or /var/lib/apt/lists/ while a process holds them. The lock is the only thing stopping two writers from corrupting the package database.

dpkg was interrupted and apt refuses to run

When dpkg is killed between unpacking a package and running its configuration script, it records that half-done state in /var/lib/dpkg/status. Every later apt command reads that state and stops, because apt will not build on a database with unfinished work in it. Whatever wording apt uses when it refuses, the first move is the same.

sudo dpkg --configure -a

This finishes the configuration step for every package that was unpacked but never configured. It runs the maintainer scripts in dependency order, and it can take a long time on a half-upgraded system, so leave it alone until it returns. If it stops on a package, it prints the package name and the script that failed. Write that name down. It is the package that stopped your upgrade, and the next section shows how to read its error.

Then let apt repair the dependencies the interruption left unmet, because some packages were upgraded and the packages they depend on were not.

sudo apt --fix-broken install

Then finish the upgrade the upgrader was in the middle of:

sudo apt update
sudo apt full-upgrade

Use full-upgrade rather than upgrade, because a release upgrade removes packages, and plain upgrade refuses to remove anything. Read the summary apt prints before you confirm. A short list of removals is normal. A list that removes ubuntu-server, systemd, openssh-server or your kernel package is not, and you should say no and find out why apt wants that before you continue.

Check the result before you do anything else:

sudo dpkg --audit
sudo apt-get check

dpkg --audit lists every package still in a broken state, and apt-get check reports unmet dependencies. Both should print nothing. When they do, run sudo apt autoremove to drop the 24.04 packages nothing depends on any more, then confirm the release with cat /etc/os-release, then run sudo update-initramfs -u -k all and sudo update-grub, and only then reboot.

How to read /var/log/dist-upgrade to find the package that stopped it

The upgrader writes everything to /var/log/dist-upgrade/. If you have run it more than once, it moves the earlier attempt's logs into a subdirectory named with a timestamp, so check ls -la /var/log/dist-upgrade/ first and read the directory that matches the failed run.

main.log is the upgrader's own diary. It records which phase the run was in and what it decided about your sources. If the upgrader itself crashed, the Python traceback is here too. Read it from the end: the last lines tell you the phase it was in when it stopped, and a traceback there means the tool failed, not a package.

apt.log holds the dependency resolver's reasoning. It is verbose, and it matters when apt refused to compute the upgrade at all, which is a failure before any package was touched. If your upgrade got as far as installing packages, you can usually skip it.

apt-term.log is the file you want for a package failure. It captures the terminal output of dpkg during the upgrade, the same text that would have scrolled past you. The last package it mentions before the end is the one that was running when the upgrade stopped, and if a maintainer script failed, dpkg's complaint about it is right there with the script's own error just above it.

sudo tail -n 60 /var/log/dist-upgrade/apt-term.log
sudo grep -in 'error' /var/log/dist-upgrade/apt-term.log | tail

Cross-check against /var/log/dpkg.log, which records every state change dpkg makes with a timestamp. tail -n 30 /var/log/dpkg.log shows the last package dpkg touched and what it was doing to it, which is the same answer from a second source.

Most failures at this stage on a server come from one of a few causes. A service the package restarts in its postinst fails to start because of a config file you customised, and systemctl status plus journalctl -xeu on that service names the line it does not accept. A disk filled up, most often /boot with old kernels or /var with apt's package cache, and df -h / /boot /var shows it; sudo apt clean empties the cache even when apt is otherwise stuck, and a disk that reports full when du says it is not has its own explanation. A package from a third-party repository the upgrader disabled depends on a library 26.04 no longer ships. A held package (apt-mark showhold) blocked a dependency from moving. Fix the cause, then run sudo dpkg --configure -a again. It picks up where it stopped.

If one package refuses to configure no matter what you do, and nothing important depends on it, remove it and reinstall it after the upgrade is complete:

sudo dpkg --remove --force-remove-reinstreq package-name
sudo dpkg --configure -a

Use that on a package you can name and explain, never on a library or anything in the ubuntu-server dependency chain, because forced removal skips the checks that would tell you what else breaks.

The sources flipped but the packages did not

The upgrader rewrites your apt sources early, before it downloads any packages. Interrupted after that point, the sources name 26.04 while the installed packages are a mix. That mix confuses the tools, and it is why do-release-upgrade may now insist that there is no new release.

Compare the two places that describe which release you are on. /etc/apt/sources.list.d/ubuntu.sources is the deb822 sources file that 24.04 introduced, and its Suites: lines carry the release codename. /etc/os-release is written by the base-files package and says which release is actually installed.

grep -E '^(Suites|Components):' /etc/apt/sources.list.d/ubuntu.sources
grep -E '^(VERSION_ID|VERSION_CODENAME)=' /etc/os-release
apt-cache policy base-files

Three combinations matter. Sources still name 24.04 (codename noble) and os-release still says 24.04: the upgrade never got past its checks, and you can run sudo do-release-upgrade again after reading main.log to learn why it stopped. Sources name the 26.04 codename and os-release still says 24.04: the package swap started and was interrupted, and the dpkg repair in the previous section, ending with apt full-upgrade, is the way to finish it. Sources name 26.04 and os-release says 26.04: base-files was one of the packages that made it through, and the system now calls itself 26.04 even if most of it is not.

That last combination is the trap. do-release-upgrade decides what release you are on from the same information os-release carries. If that already says 26.04, the tool looks for something newer than 26.04, finds nothing, and reports that no new release is available. The tool is answering a question about os-release, and os-release is wrong. Stop asking the upgrader and finish with apt: sudo apt update, then sudo apt full-upgrade, which upgrades every package still at its 24.04 version, then sudo apt autoremove. The other reasons do-release-upgrade reports no new release, such as an LTS prompt that waits for the first point release, are worth ruling out if os-release still says 24.04.

The upgrader also disables third-party sources under /etc/apt/sources.list.d/ and keeps a backup copy of each file it changed, with a suffix added to the original name. Run ls -la /etc/apt/sources.list.d/ and diff each original against its backup to see exactly what it did. Leave the third-party entries disabled until the Ubuntu packages are consistent, then re-enable each one only after confirming the vendor publishes packages for 26.04. If apt update complains that a source is configured more than once, an old sources.list entry and the new ubuntu.sources entry describe the same suite, and the deb822 duplicate source error explains which one to remove.

The server will not boot after the upgrade

A reboot is where a half-finished upgrade shows its cost. The likely causes on a VPS are a kernel installed without its initramfs, a GRUB configuration that was never regenerated, a package left half configured that a boot-time unit depends on, or a disk that filled up while dpkg was writing.

Open the provider's console before you touch anything else. It shows you where the boot stops: the GRUB menu, a kernel panic, a filesystem check waiting for an answer, or a systemd emergency shell asking for the root password. That one observation decides the next step.

If GRUB appears, boot the previous 24.04 kernel from the advanced options submenu. The old kernel normally stays installed until autoremove runs. Once the system is up on the old kernel, run sudo dpkg --configure -a and the rest of the repair from the section above, then sudo update-initramfs -u -k all and sudo update-grub before you try the new kernel again. Recovering a VPS that fails to boot after a kernel update covers the GRUB and initramfs side in detail.

If you land in an emergency shell, the root filesystem is usually mounted read-only. Remount it and run the same repair:

mount -o remount,rw /
dpkg --configure -a

If the system does not reach a shell at all, boot your provider's rescue image, mount the VPS disk, and repair from a chroot. Find the root partition with lsblk rather than guessing its name.

lsblk
mount /dev/<root-partition> /mnt
for d in dev proc sys run; do mount --bind /$d /mnt/$d; done
chroot /mnt /bin/bash
dpkg --configure -a
apt --fix-broken install
update-initramfs -u -k all
update-grub
exit
reboot

Before you spend an hour in that chroot, weigh it against the snapshot. You took one before you started, and restoring it takes minutes on most providers. Then you re-run the upgrade, which takes well under an hour on a VPS, and this time you know which package to fix in advance. Restoring is the faster path whenever any of these is true: you cannot get a console or a rescue image, you cannot name the package that stopped the upgrade, more than one package is stuck, or the box runs something people are waiting for. Patching by hand is the faster path only when you know the one thing that broke and the fix is a single command.

Copy /var/log/dist-upgrade/ off the box before you restore, from the rescue image if that is the only way in. The restore erases those logs, and a second attempt fails exactly the same way if you never learned why the first one did. What a snapshot can and cannot restore is worth reading before you rely on one. A snapshot rolls the whole disk back, including any data written since you took it, which is fine in the middle of an upgrade and not fine a week later.

Three signs the system should be rebuilt instead

Some upgrades are not worth saving. Restoring a snapshot and re-running is cheap, but if the failure came from the state of the box itself, the second run fails too, and the honest fix is a fresh 26.04 image with your data restored from backup. Three signs tell you that you are there.

First, dpkg's own database is damaged. If dpkg --audit or apt-get check fails to read /var/lib/dpkg/status at all, rather than reporting broken packages inside it, the record of what is installed is gone. Ubuntu keeps daily copies under /var/backups/ (ls -la /var/backups/dpkg.status*), and swapping the newest good copy into place sometimes works. But once that copy disagrees with what is really on disk, you are guessing, and every later apt run builds on the guess.

Second, the tools you need to repair the system are themselves broken. When apt or dpkg will not start because a shared library was removed or half replaced, or systemd cannot start units because its own package is half configured, there is no working package manager left to fix the package manager. ldd /usr/bin/apt tells you whether apt's libraries are all present. You can sometimes bootstrap out of this from a chroot in the rescue image, but the time that takes is usually longer than a rebuild.

Third, the list of broken packages is not shrinking. If you have run dpkg --configure -a and apt --fix-broken install in a loop for more than an hour, and each pass exposes a new package instead of clearing the last one, the system carried problems into the upgrade that the upgrade did not create: files under /usr edited by hand, packages pinned or held, a third-party repository that replaced core libraries, or an earlier upgrade that was itself never finished. A fresh image does not have those problems, and restoring your data onto it takes less time than finding them.

A rebuild is only cheap if the data lives somewhere other than the box. That is the difference between a snapshot and a backup, and the reason the upgrade guide asks for both.

FAQ

Can I just run do-release-upgrade again after it was interrupted?

Yes, and it is the best first move. If the upgrader is still alive in its screen session, running it again reattaches to that session. If it is not, run sudo dpkg --configure -a and sudo apt --fix-broken install first, then start the upgrader again. It re-reads the current state and continues. The one case where it cannot help is when /etc/os-release already says 26.04, because then it believes the upgrade is done; finish with sudo apt full-upgrade instead.

Why does do-release-upgrade say there is no new release after my upgrade failed?

Because base-files, the package that writes /etc/os-release, was one of the packages that upgraded before the interruption. The tool now reads that file, concludes you are on 26.04, and finds nothing newer to offer. Compare grep VERSION_ID /etc/os-release with grep Suites /etc/apt/sources.list.d/ubuntu.sources, then finish with sudo apt update && sudo apt full-upgrade.

Is it safe to reboot a half-upgraded Ubuntu server?

Not until sudo dpkg --audit returns nothing. A reboot with a kernel unpacked but not configured, or with GRUB not regenerated, is the most common way an upgrade that could have been fixed in ten minutes becomes a rescue-image job. Finish the dpkg repair and apt full-upgrade, run update-initramfs -u -k all and update-grub, and only then reboot.

How do I find which package stopped the upgrade?

Read the end of /var/log/dist-upgrade/apt-term.log, which holds dpkg's terminal output. The last package named before the log ends is the one that was running, and a failed maintainer script prints its error just above dpkg's complaint. tail -n 30 /var/log/dpkg.log confirms it from a second source. If instead main.log ends in a Python traceback, the upgrader itself crashed and no package is at fault.

Should I restore the snapshot or keep fixing?

Restore when you cannot name the package that failed, when more than one package is stuck, when you have no console access, or when the server needs to be back quickly. Keep fixing only when you know the one thing that broke and the fix is a single command. Copy /var/log/dist-upgrade/ off the box before restoring, or the second attempt will fail the same way.