SSH key basics: how to manage your keys well

How SSH keys dey work

An SSH key na pair of files: private key wey dey stay for your device, and public key wey you copy go every server wey you want log into. When you connect, the server go use the public key send challenge wey only the matching private key fit answer. The private key no ever leave your device, so no secret dey travel through network, and breached server get nothing useful to steal. Na why keys better pass passwords. To manage SSH keys well, follow four habits: one key for each device, the file permissions wey sshd demand, one ~/.ssh/config file so you no need type options every time, and knowing how to remove key the same day laptop go missing.

This guide cover each habit for Ubuntu 24.04, although almost everything here apply to any Linux server and any recent OpenSSH.

Make we clarify one vocabulary matter before we start, because e fit prevent real mistakes. Public key no be secret. You fit paste am for ticket, send am by email, or publish am, and nobody fit log in with am. Private key na the secret. Anybody wey copy that file, and know the passphrase if e get one, na you be that person as far as your servers concern.

Make key: ed25519 na the correct default

For your own computer, no be for server, run:

ssh-keygen -t ed25519 -C "laptop"

-t ed25519 dey choose the key type. Ed25519 na the modern default: the keys short, fast, and every OpenSSH release since 2014 support dem. Use ssh-keygen -t rsa -b 4096 only when you need connect to old device wey no understand ed25519. -C "laptop" dey set comment. The comment no get any cryptographic work, but na through am you go recognise this key inside server authorized_keys file two years from now. So use the name of the device wey the key dey on.

ssh-keygen dey ask where to save the key. Accept the default, ~/.ssh/id_ed25519. Then e go ask for passphrase. Set one; the passphrase section below explain why e no go cost you anything for everyday use. You go get two files: ~/.ssh/id_ed25519 na the private key, and ~/.ssh/id_ed25519.pub na the public key. Check the public half:

cat ~/.ssh/id_ed25519.pub

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF3k2s0vQx7GdKQhX1yBz... laptop

Na one line: the key type, the key material, and your comment. Na this line go end up for your servers.

One key for each device, no be one for each server

The first question wey everybody dey ask na: I need new key for every server? No. Create one key for every device wey you dey type on, then put that one public key for every server wey the device need reach. The key na the device identity. The authorized_keys file for each server na the list of devices wey get permission to enter.

Na this model dey scale well, and the other options dey fail for ways wey you fit predict. If you get one key for each server, laptop wey connect to twenty servers go carry twenty private keys, and you go lose track of which one belong to which server. One key wey all your devices share worse pass that: if person thief the laptop, you no fit revoke only the laptop without locking your desktop out too, because both devices hold the same private key. So you go need replace the key everywhere and distribute the new one to every device at once.

With one key for each device, lost laptop go cost you only one line for each server: delete the laptop line from authorized_keys, and every other device go continue to work. The comment wey you set with -C na wetin make that line easy to find.

The rule behind this model be say: you create private key for one device, and e dey end with that device. Never copy private key go another machine, and never upload private key to server. When new device need access, generate new key for that device.

Put public key for server

The easy way na ssh-copy-id, wey come with OpenSSH:

ssh-copy-id matt@10.0.0.10

E go log in with anything wey still dey work, usually password, append your public key to ~/.ssh/authorized_keys for the server, and create the directory plus the file with correct permissions if dem no dey. Test am by opening new SSH session: the server suppose let you enter without asking for the account password. If your key get passphrase, na your own machine fit ask for that instead; that prompt na local, e no be server password.

When password login don already disable, ssh-copy-id no fit enter, so you go add the line by hand. Log in through session wey still dey work, or your provider web console, then run this for the server:

mkdir -p ~/.ssh && chmod 700 ~/.ssh
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF3k2s0vQx7GdKQhX1yBz... laptop" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Paste your real public key inside the quotes, the complete single line from id_ed25519.pub. authorized_keys na one public key for each line, and na the whole access database be that: to add device, append one line; to revoke device, delete one line. For fresh server, this step suppose happen inside the first 10 minutes for new VPS, just before you turn password login off.

Permissions wey dey spoil key login

Na this be the most common way key login dey fail, and client side no dey show any clear error. sshd dey run with StrictModes yes by default for Ubuntu 24.04. This mean e no go use any authorized_keys file wey other users fit edit. If anybody apart from you fit write the file, the ~/.ssh directory, or your home directory, sshd go ignore your key and fall back to asking for password. Client no go explain why. (Ubuntu OpenSSH dey tolerate only one narrow case: file wey your own private group fit write, when nobody else dey inside that group. No depend on this; use the modes below.) The reason dey only appear for server log:

sudo grep 'Authentication refused' /var/log/auth.log

For minimal image wey no get rsyslog, auth.log no dey. The same line dey inside the journal: sudo journalctl -u ssh | grep 'Authentication refused'.

Authentication refused: bad ownership or modes for file /home/matt/.ssh/authorized_keys

To fix am, change two permissions and check ownership. Run the commands for server as the affected user:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R matt:matt ~/.ssh

The rule wey you need remember be this: 700 for the .ssh directory, and 600 for everything inside am. The same numbers apply for your own computer, because the client dey check too. If other users fit read private key, ssh go reject the key immediately. This time, e go show clear error:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/home/matt/.ssh/id_ed25519' are too open.

chmod 600 ~/.ssh/id_ed25519 go fix am.

~/.ssh/config: stop typing options

A ~/.ssh/config file for your own computer dey give every server one short name and remember the options wey you dey type often. Create am with 600 permissions, then add one Host block for each server:

Host web1
    HostName 10.0.0.10
    User matt
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Host db1
    HostName 10.0.0.11
    User matt
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Now ssh web1 go replace ssh -p 22 matt@10.0.0.10, and the same short name go work for scp, rsync, and git, because all of dem dey read this file. HostName na the real address, User save you from typing the account name, and IdentityFile pin which key to offer.

IdentitiesOnly yes deserve one sentence because e dey fix one confusing failure. When your agent hold several keys, the client go offer dem one after another, and the server go count every offer as failed attempt. If you load enough keys, you go get Received disconnect: Too many authentication failures before e ever try the correct key. IdentitiesOnly yes make the client offer only the key wey dem name for IdentityFile, so this failure no fit happen.

Passphrases and ssh-agent

Passphrase dey encrypt private key file wey dey disk. If you no use one, anybody wey copy the file fit use am immediately; if you use one, the stolen file no useful until person guess the passphrase. For key wey dey laptop, na exactly this protection you want, because person fit thief laptop and backup of laptop fit leak.

The reason passphrase no dey cost anything for practice na ssh-agent. Agent dey keep your decrypted key for memory, so you type the passphrase once for each login session, and every later connection dey happen immediately. Most desktop Linux distributions and macOS already dey run agent for you. Load your key inside am with:

ssh-add ~/.ssh/id_ed25519

ssh-add -l dey list the keys wey agent dey hold currently. One thing to note: agent forwarding (ssh -A) allow remote server use your agent to authenticate to another server while you dey connected, so only enable am toward servers wey you trust well, and leave am off by default.

Rotating and revoking: the lost laptop drill

To revoke a plain SSH key, na to remove the key line from authorized_keys for every server wey get am. You no need notify any certificate authority, and you no need wait for expiry date. As soon as the line comot, new logins with that key go fail.

Run this drill now, while e no be emergency. Choose one server, open ~/.ssh/authorized_keys, and find the key by the comment wey dey beside am. Use editor delete the line, or filter am out by comment:

grep -v ' laptop$' ~/.ssh/authorized_keys > ~/.ssh/authorized_keys.tmp
mv ~/.ssh/authorized_keys.tmp ~/.ssh/authorized_keys

Then confirm from the device wey you just revoke say login don fail now. Also confirm from another device say login still dey work. Remember this detail: removing a key no close sessions wey already dey open, because system only checks the key when login happen. If you dey revoke a stolen device, also check who for the server and end any session wey you no recognise.

Rotation na the same operation, but you go do the steps for another order: generate new key for the device, install am with ssh-copy-id, confirm say the new key fit log in, then delete the old line. Do this when device change hands, when key fit don expose, or when person leave team. Doing am by hand across two servers dey okay. Across twenty servers, automation better handle the work, and managing multiple Linux servers show how to push the same authorized_keys state go the whole fleet.

Wetin you no suppose do

• No share one private key across all your devices. E go make revoking one stolen device impossible unless you replace the key everywhere.

• No commit private key to git repository, even if na private one. Automated scanners dey monitor public repositories and dey try leaked keys within minutes after push. If repository later become public, e go leak the whole history.

• No upload your laptop private key to server so that server fit reach another server. Generate separate key for the server itself, then authorise that key exactly where e dey needed.

• No paste private key for chat, email, or ticket. The public key, the .pub file, na the only half wey anybody ever share.

Once your key dey log you in reliably, take the next step and turn password authentication off. This one go stop constant guessing against your server from succeeding at all. You fit find the drop-in configuration for SSH hardening for VPS.

FAQ

SSH keys dey work how without sending password?

Server dey keep your public key for ~/.ssh/authorized_keys. When you dey log in, e go send challenge. Your client go sign the challenge with private key, and server go verify the signature with public key. Private key no ever leave your device, so nobody fit intercept am for transit, and nobody fit steal reusable credential from server. If server get breached, e go leak only public keys, and dem no fit use dem log in anywhere.

I suppose use the same SSH key for all my servers?

You fit use one key for many servers, as long as that key dey only one device. Rule na one key per device, no be one per server: your laptop public key go dey every server wey the laptop need access to, while your desktop get im own key. This one make revocation simple, because if device lost, you just remove one identifiable line from each server, and other devices go continue to work.

Which permissions .ssh directory and authorized_keys suppose get?

Set 700 on ~/.ssh, and 600 on authorized_keys plus every private key. The account wey dey use dem suppose own dem. sshd dey run with StrictModes yes by default, so if anybody apart from you fit write to file or home directory, e go silently ignore your key. The only trace go be Authentication refused: bad ownership or modes for server auth log or journal.

How I go remove SSH key from server?

Delete the key line from ~/.ssh/authorized_keys for the account wey you authorise am for. Find the correct line through the comment, wey be the label after the key material. New logins with that key go fail immediately, but sessions wey already open go remain open. So if person steal the device, end any live session for that device too. Repeat this for every server wey you copy the key go.

I need passphrase for my SSH key?

For key wey dey laptop or desktop, yes. Passphrase dey encrypt the key file, so if person steal or leak copy, e useless by itself. Also, ssh-agent mean say you type am once per session instead of every connection. Keys wey unattended automation dey use for server usually no get passphrase, because no human dey there to type am. Protect those keys by limiting wetin the target account fit do.