What to Do If Your VPS Is Hacked

Do not clean a hacked VPS

If your VPS is hacked, the decision that matters most comes before you run any command. Do not try to clean the machine. Isolate it at the provider, snapshot the disk as evidence, rotate every credential it held, then rebuild on a fresh server from sources you trust.

You cannot prove a rootkit is gone, because the tools that would prove it are the tools the attacker controls.

That is the whole argument. Here is the mechanism behind it. An attacker who reached root can replace ps so that one process ID never appears in its output. One line in /etc/ld.so.preload loads attacker code into every dynamically linked program on the box, so ls, ss and find all lie in the same consistent way. A loadable kernel module can hide files below the system call, so even a freshly downloaded binary sees a clean disk. You delete the miner, the CPU graph drops, and the server goes quiet. Quiet is also what a working backdoor looks like.

Rebuilding costs less than it feels like it costs. A typical VPS is a handful of packages, one config directory and one data set, so a rebuild is a finite job with an end. Hunting for every change an attacker made is open ended, and it never reaches proof.

Confirm that it is really a compromise

Many servers reported as hacked are not. Thousands of failed SSH logins a day are internet background noise, because every public IPv4 address is scanned continuously. A lastb output full of root and admin attempts means the scanners found your port. It does not mean anyone got in.

These signals do mean something:

• A successful login you cannot account for, such as Accepted password for root from 203.0.113.7.
• A key in authorized_keys that you did not add.
• An abuse notice from your host about traffic leaving your server.
• A process at 100% CPU carrying a name copied from a kernel thread. Miners dropped through exposed Redis and Docker sockets are commonly reported under names such as kdevtmpfsi and kinsing.
• Outbound connections to addresses none of your services use.

The kernel thread disguise has a quick test. Real kernel threads print inside square brackets and have no executable behind them, so sudo ls -l /proc/<pid>/exe fails for them with No such file or directory. If a process printed as [kworker/0:2] has an exe link pointing at something under /tmp, it is an ordinary user program wearing a kernel name.

Run these checks knowing the box may be lying to you. They are enough to decide that something is wrong. They are not enough to decide that nothing is.

Cut the network at the provider, not from inside the box

Isolation comes first, because every step after it is wasted while somebody else still holds a shell. Reading logs, rotating keys and restoring data are all pointless with a live attacker watching.

Do it in your provider control panel, in the network firewall that runs outside your operating system. Deny inbound and outbound, and keep the web console as your way in. Rules enforced there survive anything happening on the disk.

There are two reasons not to do this from inside the server. A firewall you configure inside a compromised kernel is enforced by that kernel, and root can flush nftables as easily as you can write it. And sudo ip link set enp1s0 down over SSH cuts your own session first, which locks you out of a machine you were in the middle of examining.

Block outbound as well as inbound. A reverse shell dials out from your box to the attacker, so an inbound-only block leaves an established connection working perfectly. If your provider offers inbound rules only, the remaining options are detaching the network interface or powering the instance off.

Do not reboot yet. Check whether /var/log/journal exists first. If that directory is absent, journald is writing to /run/log/journal, which lives in memory, so a reboot deletes the record of the intrusion. Running processes disappear at reboot too, and their command lines are often the clearest evidence you will ever get.

Snapshot the disk before you touch anything

A snapshot and a backup do different jobs here. The snapshot you take now is a copy of a compromised disk: it is your evidence, and it is the only thing that lets you go back after you overwrite something by accident. Your older backups are the recovery path. If your provider's panel uses the two words loosely, read how VPS snapshots differ from real backups first, because the retention rules and the restore behaviour are not the same.

Take the snapshot from the provider panel before you log in again. A live snapshot is crash consistent: it captures the disk as it was at that instant, the same as pulling the power. That is fine for evidence. Name it so that nobody restores it by mistake. Something as blunt as COMPROMISED-do-not-restore-2026-08-12 is the right level of subtlety. Keep it until your investigation is finished and any abuse ticket with your host is closed.

How to get in when SSH is gone

Two paths, both in the provider panel. The web console (VNC or serial) attaches to the machine as if you had plugged in a keyboard. It works when sshd is dead, when the firewall is wrong and when the attacker changed the SSH port. It authenticates with a local password, so a key-only server may need a root password reset before the console is useful.

Rescue mode is the better option. It boots a small live system with your disk attached but not running, so your commands are trustworthy: the compromised kernel and the compromised binaries are not executing. Mount the disk read only.

lsblk -f
sudo mkdir -p /mnt/victim
sudo mount -o ro /dev/vda1 /mnt/victim

If lsblk shows LVM (logical volume manager) volumes instead of a plain partition, activate them with sudo vgchange -ay first, then mount the device that appears under /dev/mapper/.

Do not chroot into the mounted disk to look around. A chroot executes the attacker's binaries with your permissions, which throws away the entire reason you booted rescue mode.

Collect the evidence that is still trustworthy

Run these from rescue mode, with the disk mounted read only at /mnt/victim. Start with logins, because they date the intrusion, and everything else is easier once you have a time window.

sudo grep -aE 'Accepted (password|publickey)' /mnt/victim/var/log/auth.log
sudo last -f /mnt/victim/var/log/wtmp
sudo lastb -f /mnt/victim/var/log/btmp
sudo journalctl -D /mnt/victim/var/log/journal -u ssh --since "2026-07-01"

A missing /var/log/auth.log is not itself suspicious. Some current Ubuntu images ship without rsyslog, so sshd logs to the journal only, which is what the journalctl -D line reads. What is worth noting is a gap in otherwise continuous logs, or a log file truncated to zero bytes. Log wiping is common and usually clumsy.

Next the accounts and the keys.

sudo find /mnt/victim/root /mnt/victim/home -name 'authorized_keys*' -exec ls -l {} +
sudo awk -F: '$3 == 0 {print $1}' /mnt/victim/etc/passwd
sudo lsattr /mnt/victim/root/.ssh/authorized_keys

The awk line prints every account with user ID 0. Anything other than root in that output is a second root account. The find pattern deliberately matches authorized_keys2 as well, because OpenSSH reads both file names by default and the second one is easy to overlook. If lsattr prints an i in the attribute list, the file is immutable: an attacker sets that flag so your attempt to delete their key fails with Operation not permitted, and a tired admin assumes the edit worked.

Persistence hides in a small number of places, so check all of them.

sudo ls -lt /mnt/victim/etc/systemd/system
sudo ls -l /mnt/victim/etc/cron.d /mnt/victim/var/spool/cron/crontabs
sudo cat /mnt/victim/etc/ld.so.preload
sudo grep -rnE 'curl|wget|base64|/dev/tcp' /mnt/victim/etc/update-motd.d /mnt/victim/etc/rc.local /mnt/victim/root/.bashrc /mnt/victim/root/.profile

/etc/ld.so.preload does not exist on a normal Ubuntu or Debian system, so No such file or directory is the healthy result and any content at all deserves your attention. A login file that pipes base64 -d output into a shell is the same story: legitimate config does not need to hide its own text.

Build the timeline from change time rather than modification time.

sudo find /mnt/victim -xdev -newerct '2026-08-01' -type f -printf '%TF %TT %p\n' | sort

touch sets modification time to any value the attacker wants, so mtime lies cheaply. Change time (ctime) updates on any change to the inode, and touch cannot move it backwards, so -newerct gives a more honest list of what was written recently. It is still not proof, because root can move the system clock or write to the block device directly.

Package integrity is worth one command and one caveat. On a live system sudo dpkg --verify prints a line for every packaged file whose checksum no longer matches, with a 5 in the checksum column, and sudo debsums -ac does the same job including config files when the debsums package is installed. Read the result in one direction only. A changed /usr/sbin/sshd is real evidence. A clean report proves nothing, because the same root account that replaced the binary can rewrite the checksum lists under /var/lib/dpkg/info/. Rootkit scanners such as rkhunter and chkrootkit follow the same rule: a hit is information, a clean run is not a clearance.

Copy what you have collected off the machine before you do anything destructive.

sudo tar --ignore-failed-read -C /mnt/victim -czf /root/evidence-2026-08-12.tgz \
  var/log etc root/.ssh root/.bash_history home
sha256sum /root/evidence-2026-08-12.tgz

Write that hash down somewhere off the server. If this ever becomes an insurance claim or a police report, being able to show the archive has not changed since collection is the difference between evidence and a folder of files. Deleting things by accident during an investigation is normal, and the snapshot plus this archive is what makes it survivable. Undoing a bad rm afterwards is far harder than people expect, as recovering files deleted with rm -rf explains.

Find the door they came through

A rebuild that does not close the entry route gets you compromised again, often within days, because the scan that found you the first time never stops running. Four doors cover most single-server compromises.

SSH password login. An Accepted password for root line from an address you do not recognise is the answer on its own. Check PasswordAuthentication in /etc/ssh/sshd_config and in every file under /etc/ssh/sshd_config.d/. sshd uses the first value it obtains for a keyword, and the Include line sits at the top of the main file on Ubuntu, so a dropped-in config file quietly wins over the setting you edited further down.

A service published with no authentication. Redis on 6379, the Docker API on 2375, a database bound to 0.0.0.0 instead of 127.0.0.1. Docker is the common surprise. Publishing a container port inserts DNAT (destination network address translation) rules that are evaluated before ufw's chains, so ufw status can report a port as blocked while the container behind it answers the whole internet. Understand that before the rebuild: why Docker published ports bypass ufw covers the rule ordering and the fix.

An unpatched web application. Search the web server access log around your earliest suspicious timestamp for a POST to an upload path or an admin path, then look for files under the web root with a matching change time. A stray PHP file in an uploads directory is the classic result.

A leaked credential. A key committed to a repository, a token pasted into a chat, an .env file served as a static file by a misconfigured web server. Automation makes this easy to do by accident, which is the argument for keeping secrets out of AI agents and their config files.

If you cannot name the door after all of that, assume a leaked credential and treat every secret the machine held as public.

Rotate every credential the machine could see

Rotate after the network is cut, never before. Rotating while the attacker still has a connection simply hands over the new secrets.

• Every SSH private key stored on the server, plus every account elsewhere that trusted the matching public key.
• Any key you forwarded into the box with ssh -A. Agent forwarding leaves a socket under /tmp, and root on that machine can use it to authenticate as you anywhere your key is accepted, for as long as your session stays open.
• API tokens in .env files, in systemd Environment= lines, in CI configuration and in provider credentials.
• Database passwords, and the application accounts that use them.
• TLS (transport layer security) private keys the server held. Reissue the certificate and revoke the old one.
• Your hosting account password, with two-factor authentication turned on. That panel can rebuild, snapshot and console into every server you own, so it is the real perimeter.
• Any password typed into a shell session on that host while it was compromised, because root can record a terminal session as it happens.

If a password on that machine is used anywhere else, change it there too. Reuse is how one compromised VPS becomes a compromised email account.

The rebuild checklist

1. Create a new server from a fresh distribution image. Not from the snapshot of the compromised box, and not from a whole-root-filesystem restore.
2. Install packages from the distribution repositories. Never copy a binary across from the old disk.
3. Restore data only, from a backup dated before the earliest evidence in your timeline. Database dumps, uploads, application state. Leave /etc, /usr and the old unit files behind.
4. Enter the rotated secrets by hand. Do not copy the old .env across.
5. Inspect restored web content for files added inside the intrusion window before you serve any of it again.
6. Harden before you expose it: key-only SSH, a non-root working account, a default-deny inbound firewall, and no service published wider than it needs to be. Work through the first ten minutes on a new VPS, then harden SSH properly, then add fail2ban on Ubuntu 24.04 to cut the login noise. Give each service its own least-privilege account so the next foothold is not a root foothold.
7. Power off the old server, and keep its snapshot until the investigation and any abuse ticket are closed.
8. Fix the backups. If step 3 was guesswork, the real lesson is that your backup history was too short to reach behind the intrusion. Versioned off-server backups with long retention are what buy a clean restore point next time: restic backups on a VPS gives you both.

If you cannot date the intrusion, you cannot choose a safe backup. In that case restore only data you can inspect by eye: a SQL dump you can read, a directory of images you can list. Treat everything executable as suspect and install it again from the repositories.

What the abuse notice from your host means

Most people learn their server is compromised from their provider, not from their own monitoring. Hosts see the outbound traffic: SSH brute force against other networks, spam on port 25, or a share in a reflection attack. The ticket normally carries timestamps, ports and a sample of the flows, along with a deadline measured in hours.

Reply to it, even if your only answer is that the server is isolated and being rebuilt. Providers null-route or suspend a server when a ticket goes unanswered, which turns your incident into an outage. Then ask for the raw log lines behind the report. Those timestamps were recorded outside your machine, so they are the one part of the timeline the attacker could not edit, and they often date the intrusion better than anything on the disk.

A compromised customer server is routine work for a host, and handling one well is not held against you. The wider question of whether VPS hosting is safe mostly comes down to what the customer configures, which is exactly the part you now get to do again from scratch.

When to call a professional

• The server held personal data belonging to other people. Under the GDPR (General Data Protection Regulation), a personal data breach must be reported to the supervisory authority without undue delay, and within 72 hours of becoming aware of it where that is feasible. Judging whether that clock has started is legal work, not sysadmin work.
• Payment card data was in scope. The card schemes require an approved forensic investigator, and your own poking around can damage the case.
• There is an extortion demand, or your data has been encrypted.
• The machine could reach other machines: an internal network, a hypervisor, a CI runner holding production credentials. One compromised host in a group is a group-wide incident until proven otherwise.
• You will need the evidence to hold up for insurance or law enforcement. Stop at the snapshot, take a full disk image, and record who handled it and when.

For a single VPS running your own services, with nobody else's data on it, the playbook above is the whole job. Isolate at the provider. Snapshot for evidence. Collect what is still trustworthy. Rotate everything. Rebuild clean.

FAQ

Can I clean a hacked VPS instead of rebuilding it?

Not with any confidence, because you would be asking the compromised system to report on itself. A replaced ps hides a process, a line in /etc/ld.so.preload injects code into every dynamically linked tool you run, and a kernel module can hide files from every program at once. You can find things, so a hit is meaningful. You cannot demonstrate absence, so a clean result is not. Cleaning is defensible only when the server holds nothing you care about and you accept that it may be compromised again.

Should I power off a compromised server or leave it running?

Cut its network at the provider first, then leave it running long enough to take a snapshot and look at the running processes. Powering off destroys the process list, and it deletes the journal entirely when /var/log/journal does not exist, because journald then writes to memory under /run. Power it off anyway if it is actively attacking other networks and you have no way to block its outbound traffic. Stopping the harm outranks keeping the evidence.

How do I work out when the attacker got in?

Find the earliest Accepted password or Accepted publickey line you cannot explain, in /var/log/auth.log or in the journal. Cross-check it with a change-time listing, find / -xdev -newerct 'YYYY-MM-DD' -type f, since ctime is harder to forge than mtime. Then compare both against the timestamps in your provider's abuse ticket, which were recorded off the machine and could not be edited. Pick a backup older than the earliest of those three dates. If nothing lines up, assume the compromise is older than your backup history and restore only data you can inspect.

Are my backups safe to restore after a compromise?

The data usually is, with inspection. The system files are not. A backup taken after the intrusion contains the backdoor, so restoring a whole root filesystem restores the attacker as well. Check the backup repository itself too: if the credentials for it were stored on the compromised server, the history may have been deleted or altered, which is the argument for append-only or pull-based backup targets. Restore application data, then install the software again from the distribution repositories.

Do I have to tell anyone that my VPS was compromised?

Always reply to your host's abuse notice. Beyond that it depends on whose data was on the machine. Personal data belonging to other people can trigger a legal reporting duty, such as the GDPR's 72-hour notification to a supervisory authority. If user credentials were stored on the server, tell those users so they can change passwords elsewhere. If keys on the box authorised access to third-party systems, such as a code host or a cloud account, tell those providers so they can check for misuse. A purely personal server holding nobody else's data carries no obligation beyond the abuse ticket.