how to manage SSH keys well

How SSH keys work

SSH key na pair of files: private key wey go stay for your device, and public key wey you go copy go every server wey you want log in to. When you connect, the server go use the public key send challenge wey only the matching private key fit answer. The private key no dey leave your device, so no secret dey travel over the network, and if dem hack one server, dem no go get anything useful to steal. Na why keys better pass passwords. To manage SSH keys well, you need four habits: one key per device, the file permissions wey sshd demand, a ~/.ssh/config file so you no go dey type options every time, and how to remove one key immediately laptop lost.

This guide go cover each habit for Ubuntu 24.04, even though almost everything for here apply to any Linux server and any recent OpenSSH.

One thing wey you must know before we start, because e dey prevent big mistake. The public key no be secret. You fit paste am for ticket, send am for email, or publish am, and nobody fit use am log in. The private key na the secret. Anybody wey copy that file, and know its passphrase if e get one, na you dem go see as far as your servers concern.

Create a key: ed25519 na di correct default

For your own computer, no be for server, run:

ssh-keygen -t ed25519 -C "laptop"

-t ed25519 dey pick di key type. Ed25519 na di modern default: di keys dey short, dem dey fast, and every OpenSSH release from 2014 dey support dem. Use ssh-keygen -t rsa -b 4096 only if you must connect to old device wey no understand ed25519. -C "laptop" dey set comment. Di comment no dey do any cryptographic work, but na how you go take recognize dis key inside server authorized_keys file two years from now, so use di name of di device wey di key dey for.

ssh-keygen dey ask where you wan save di key. Accept di default, ~/.ssh/id_ed25519. Den e go ask for passphrase. Set one; di passphrase section below go explain why e no go cost you anything for everyday work. You go get two files: ~/.ssh/id_ed25519 na di private key, and ~/.ssh/id_ed25519.pub na di public key. Look at di public half:

cat ~/.ssh/id_ed25519.pub

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF3k2s0vQx7GdKQhX1yBz... laptop

E na one line: di key type, di key material, and your comment. Na dat line wey go end up for your servers.

One key for every device, no be for every server

The question wey everybody dey ask first: I need new key for every server? No. Create one key for every device wey you dey use, and put that one public key for every server wey the device need to reach. The key na him dey identify the device. The authorized_keys file for each server na the list of devices wey dey allowed enter.

This na the model wey dey work well, and the other ways dey fail for ways wey easy to see. One key for every server means laptop wey get twenty servers go carry twenty private keys, and you go lose track of which one na which. One key wey all your devices dey use together bad pass: if dem steal the laptop, you no fit revoke the laptop without locking out your desktop too, because dem carry the same private key, so you go must replace the key everywhere and redistribute am to every device at once.

With one key for every device, the lost laptop go cost you only one line for every server: delete the laptop line for authorized_keys, and every other device go still work. The comment wey you set with -C na him go make that line easy to find.

The rule for this model: private key dey start for one device and e go die with that device. No ever copy private key to second machine, and no ever upload any one to server. When new device need access, generate new key for am.

Put the public key for server

The easy way na ssh-copy-id, wey come with OpenSSH:

ssh-copy-id matt@10.0.0.10

E go log in with whatever still dey work, usually password, e go add your public key for ~/.ssh/authorized_keys for server, and e go create the directory and the file with correct permissions if dem no dey. Test am by opening new SSH session: the server suppose let you enter without ask for account password. If your key get passphrase, your own machine fit ask for dat one instead; dat prompt na local matter and no be the server password.

When password login don already disabled, ssh-copy-id no fit enter, so you must add the line by hand. Log in through session wey still dey work, or your provider web console, and run dis for server:

mkdir -p ~/.ssh && chmod 700 ~/.ssh
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF3k2s0vQx7GdKQhX1yBz... laptop" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Paste your real public key inside di quotes, di full single line from id_ed25519.pub. authorized_keys na one public key per line, and na dat one be di entire access database: to add device, you just add one line, and to remove device, you just delete one line. For fresh server, dis step dey inside di first 10 minutes for new VPS, right before you turn password login off.

The permissions wey dey break key login

Dis na di most common reason why key login dey fail, and e dey fail silently for client side. sshd dey run with StrictModes yes by default for Ubuntu 24.04, wey mean say e no go use authorized_keys file wey other users fit edit. If anybody fit write for di file, di ~/.ssh directory, or your home directory, sshd go ignore your key and e go start to ask for password, without any explanation for client. (Ubuntu's OpenSSH dey allow only one small case: a file group writable by your own private group, wey nobody else dey inside. No rely on am; follow di modes wey dey below.) Di reason go only show for server log:

sudo grep 'Authentication refused' /var/log/auth.log

If you dey use minimal image wey no get rsyslog, you no go see auth.log; di same line dey inside di journal: sudo journalctl -u ssh | grep 'Authentication refused'.

Authentication refused: bad ownership or modes for file /home/matt/.ssh/authorized_keys

Di fix na two permission changes and one ownership check, run am for server as di user wey dey affect:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R matt:matt ~/.ssh

Di rule wey you must remember: 700 for di .ssh directory, and 600 for everything inside am. Di same numbers apply for your own computer, because client dey check am too. If other users fit read your private key, ssh go refuse di key completely, and dis time di error go loud:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/home/matt/.ssh/id_ed25519' are too open.

chmod 600 ~/.ssh/id_ed25519 go fix am.

~/.ssh/config: stop typing options

One ~/.ssh/config file for your own computer dey give every server short name and e dey remember all the options wey you dey type. Create am with 600 permissions and add one Host block for every server:

Host web1
    HostName 10.0.0.10
    User matt
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Host db1
    HostName 10.0.0.11
    User matt
    Port 2222
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Now ssh web1 go replace ssh -p 22 matt@10.0.0.10, and that same short name go work for scp, rsync, and git, because dem all dey read this file. HostName na the real address, User dey help you avoid to type the account name, and IdentityFile dey tell am which key to use.

IdentitiesOnly yes need explanation, because e dey fix one confusion wey dey cause failure. When your agent get many keys, the client go dey offer dem one by one, and the server go count every single offer as one failed attempt. If you load many keys, you go see Received disconnect: Too many authentication failures before the client even try the correct key. IdentitiesOnly yes dey make the client offer only the key wey you name for IdentityFile, so that failure no go fit happen.

Passphrases and ssh-agent

Passphrase dey encrypt your private key file for disk. If you no use one, anybody wey copy the file fit use am sharp-sharp; if you use one, the stolen file no go work until dem guess the passphrase. For key wey dey laptop, na that kind protection you need, because laptop dey stolen and laptop backup dey leak.

The reason why passphrase no dey cost anything for practice na ssh-agent. The agent dey hold your decrypted key for memory, so you go type the passphrase once per login session and every connection wey follow am go dey instant. Most desktop Linux distributions and macOS don already run agent for you. Load your key into am with:

ssh-add ~/.ssh/id_ed25519

ssh-add -l go list all the keys wey the agent dey hold now. One caution: agent forwarding (ssh -A) dey allow the remote server use your agent to authenticate onward while you dey connect, so only enable am for servers wey you trust well-well, and leave am off by default.

Rotating and revoking: the lost laptop drill

To revoke one plain SSH key, you just need to remove its line from authorized_keys for every server wey get am. You no need to notify any certificate authority and you no need to wait for expiry date. As soon as that line don commot, new logins with that key go fail.

Run this drill now, before emergency start. Pick one server, open ~/.ssh/authorized_keys, and find the key by its comment. Use editor delete the line, or use filter for the comment:

grep -v ' laptop$' ~/.ssh/authorized_keys > ~/.ssh/authorized_keys.tmp
mv ~/.ssh/authorized_keys.tmp ~/.ssh/authorized_keys

After dat, confirm from the device wey you just revoke say login fail, and from another device say login still dey work. Note one thing: to remove key no dey close sessions wey don open already, because system dey check key only when person dey login. If you dey revoke stolen device, check who for the server too and kill any session wey you no recognize.

Rotation na the same operation but the order different: generate new key for the device, install am with ssh-copy-id, confirm new key fit login, then delete the old line. Do dis when person dey change device, when key fit don leak, or when person leave team. To do dis manually for two servers dey okay; but for twenty servers, you need automation, and managing multiple Linux servers go show you how to push the same authorized_keys state to many servers at once.

Wetin you no suppose do

• No use one private key for all your devices. If one device loss, you no go fit revoke am without change the key for everywhere.

• No commit private key inside git repository, even if the repo private. Automated scanners dey watch public repositories and dem dey try leaked keys within minutes of a push, and if you change repository to public later, the whole history go leak.

• No upload your laptop private key to server just so that server fit reach another server. Generate new key for the server itself, and authorize that key only for where you need am.

• No paste private key for chat, email, or ticket. The public key, the .pub file, na only that one wey you suppose share.

Once your key dey work well for login, do the next step and turn password authentication off. This way, dem no go fit continue to guess your server password. The configuration for that one dey SSH hardening on a VPS.

FAQ

How SSH keys dey work without send password?

Server dey keep your public key inside ~/.ssh/authorized_keys. When you wan login, server go send challenge, your client go sign that challenge with private key, then server go use public key verify the signature. Private key no dey ever leave your device, so nobody fit intercept anything for transit and nothing for steal from server wey dem fit reuse. If dem hack server, na only public keys dem go leak, and dem no fit use dem log in anywhere.

I suppose use the same SSH key for all my servers?

Using one key for many servers na correct thing, as long as that key dey stay for one single device. The rule na one key per device, no be one per server: your laptop public key must dey every server wey laptop need, and your desktop must get its own key. This way, to revoke key go easy, because if you lose one device, you go just remove one line for every server, and other devices go still dey work.

Wetin be the permissions for .ssh directory and authorized_keys?

Set 700 for ~/.ssh and 600 for authorized_keys and for every private key, and the account wey dey use dem must own dem. sshd dey run with StrictModes yes by default, so if any person wey no be you fit write for file or home directory, sshd go just ignore your key without tell you, and the only trace na Authentication refused: bad ownership or modes for server auth log or journal.

How I go remove SSH key from server?

Delete the line for that key from ~/.ssh/authorized_keys for the account wey dem authorize. Find the correct line by the comment wey dey follow am, na the label wey dey after the key material. New login with that key go fail immediately, but sessions wey don open already go still dey open, so if the device stolen, make you end any live session for that device too. Do this same thing for every server wey you copy the key to.

I need passphrase for my SSH key?

If the key dey laptop or desktop, yes. Passphrase dey encrypt the key file, so if dem steal or leak the copy, e no go useful by itself, and ssh-agent means you go type am once per session instead of every connection. Keys wey unattended automation dey use for server usually no get passphrase, because no human dey there to type am; protect those ones by restrict wetin the target account fit do.