SSH hardening on a VPS

Why SSH is the first thing to harden

SSH is how you control your server, which makes it the lock every attacker tries first. The moment a VPS is online, scanners start guessing usernames and passwords on port 22. You can watch it happen in your logs within minutes. Hardening SSH is about removing the things they can guess: turn off password login entirely, turn off root login, and let only cryptographic keys in. Once you do, the constant guessing simply cannot succeed, because there is no password to find.

This assumes you already have SSH working. If you can log in, you can harden it. Do the steps in order and keep your current session open until a new one works, so a mistake never locks you out.

Step 1: Make sure key authentication works first

Key authentication replaces a password with a key pair: a private key that stays on your computer and a public key you put on the server. The server proves you hold the private key without it ever leaving your machine. Before you disable passwords, confirm keys work, or you will lock yourself out.

On your own computer, create a key if you do not have one:

ssh-keygen -t ed25519

Copy the public half to the server:

ssh-copy-id user@your-server

Then open a new SSH session. If it lets you in without asking for a password, your key works and you are safe to turn passwords off.

Step 2: Harden sshd with a drop-in file

Do not edit /etc/ssh/sshd_config directly. Ubuntu 24.04 reads drop-in files from /etc/ssh/sshd_config.d/, and a small file there is cleaner, survives package upgrades, and is easy to remove if something goes wrong. The name matters: sshd keeps the first value it reads for each setting, and Ubuntu cloud images ship 50-cloud-init.conf with PasswordAuthentication yes in this directory. Name your file 00- so it sorts before that one and wins; a 99- file loses silently. Create one:

sudo nano /etc/ssh/sshd_config.d/00-hardening.conf

Put this in:

# Key-only login: no passwords to guess.
PasswordAuthentication no
KbdInteractiveAuthentication no

# No direct root login. Log in as your user, then use sudo.
PermitRootLogin no

Each line closes a door. PasswordAuthentication no is the big one: with passwords off, a brute-force attack has nothing to brute-force. KbdInteractiveAuthentication no shuts a second password-style path. PermitRootLogin no means an attacker must know your username and hold your key, not just target the one account, root, that exists on every box.

Step 3: Test the config, then reload

Check the config for mistakes before applying it, so a typo cannot break the service:

sudo sshd -t

If it prints nothing, the config is valid. Reload SSH:

sudo systemctl reload ssh

Then check the settings sshd actually uses, so you catch a drop-in that lost to another file:

sudo sshd -T | grep -Ei 'passwordauthentication|permitrootlogin'

Both should say no. Now, without closing your current session, open a brand new one from another terminal. If it logs you in with your key, you are done. If anything is wrong, your first session is still open to fix it. This overlap is the safety net, so never skip it.

Step 4: The optional non-standard port

Moving SSH off port 22 to something like 2222 does not make it more secure in any real sense, because a determined attacker scans all ports. What it does is cut the log noise, since most automated scanners only try 22. If you want it, add Port 2222 to your drop-in file, allow the new port in the firewall first, then run sudo systemctl daemon-reload && sudo systemctl restart ssh.socket and connect with ssh -p 2222. On Ubuntu 24.04, ssh.socket owns the listening port, so a plain reload ssh leaves sshd on 22; restarting the socket is what picks up the new port. Treat it as tidiness, not protection.

Step 5: Layer the further defences

Hardened SSH keys are the foundation, and two more layers sit on top of them.

Fail2ban watches your logs and bans addresses that keep failing, which trims scanner noise and evicts them early. It pairs naturally with key-only auth: see Fail2ban on Ubuntu to stop SSH attacks.

Stronger still is keeping SSH off the public internet entirely. If you put SSH behind a WireGuard VPN and firewall port 22 to the tunnel, nobody off the VPN can even reach it, and brute-force guessing stops being possible rather than just difficult. All of this assumes a default-deny firewall underneath, which is UFW set up on the VPS.

SSH is one line of a larger checklist: the first 10 minutes on a new VPS puts the steps in order, and automatic security updates on Ubuntu keep the box patched afterwards.

FAQ

How do I disable password login for SSH on Ubuntu 24.04?

Create a drop-in file at /etc/ssh/sshd_config.d/00-hardening.conf (the 00 prefix makes it sort before 50-cloud-init.conf, whose PasswordAuthentication yes would otherwise win, because sshd keeps the first value it reads) containing PasswordAuthentication no and KbdInteractiveAuthentication no, run sudo sshd -t to check it, then sudo systemctl reload ssh. Confirm key login works in a new session before you rely on it. Editing a drop-in rather than sshd_config survives package upgrades and is easy to undo.

Should I disable root login over SSH?

Yes. Set PermitRootLogin no so nobody can log in directly as root. Log in as your normal user and use sudo for admin tasks. Root exists on every Linux box, so leaving it reachable hands an attacker a known username to target. Disabling it means they must know your account name and hold your key.

Does changing the SSH port make my server more secure?

Not meaningfully. Moving off port 22 hides you from lazy scanners that only probe 22, which cuts log noise, but a real attacker scans every port and finds it anyway. Key-only authentication is what actually stops break-ins. If you change the port, open the new one in the firewall first, then run sudo systemctl daemon-reload && sudo systemctl restart ssh.socket; on Ubuntu 24.04 the socket owns the listener, and a plain reload leaves sshd on port 22.

Do I need Fail2ban if I use SSH keys?

It is optional but still useful. With key-only authentication, password guessing cannot succeed, so Fail2ban is not what keeps attackers out. It rate-limits repeated failures from one address, which trims scanner noise from your logs and evicts repeat offenders early; a slow, distributed attack stays under its ban threshold anyway. Run it on top of key auth, and ideally keep SSH behind a VPN.

How do I recover if I lock myself out of SSH?

Use your provider's web console, which reaches the server over a serial or VNC connection that does not go through SSH. From there you can log in, fix the sshd drop-in file, and reload the service. This is exactly why you test a new SSH config in a second terminal before closing your first session, and why key authentication should already be working before you turn passwords off.