Every sysadmin knows the drill. You log into a dozen servers a day, each with a password you half-remember. Some are memorable; some you've reset three times this month because you wrote them on a sticky note and lost the note. And every time you type that password, you wonder: is this secure? Should I be using something better?

The answer is yes. Key-based authentication is not just better — it's the baseline for any serious server management. Passwords can be brute-forced. Keys (used correctly) cannot. Here's how to set it up, the mistakes people make, and the one thing most guides forget to mention.

Why Keys Beat Passwords Every Time

A strong password is still guessable given enough time and compute. A 4096-bit RSA key or an Ed25519 key? The math doesn't work that way. The computational effort to brute-force a proper SSH key exceeds the heat death of the universe.

But keys aren't magic. A misconfigured key setup can be worse than a password — you hand someone a master key with no door lock. That's why the setup matters more than the algorithm choice.

Step 1: Generate Your Key Pair 

ssh-keygen -t ed25519 -C "your_email@example.com"

Ed25519 is the current recommendation. It's faster than RSA, shorter keys for equivalent security, and less prone to implementation bugs. If you're working with legacy systems that don't support it yet, fall back to RSA 4096:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

The -C flag adds a comment (usually your email) so you can identify the key later. Don't skip this — you'll thank yourself when you have twenty keys and need to figure out which one belongs to which server.

When prompted for a passphrase, use one. A key without a passphrase is a master key with no lock. Yes, it's slightly inconvenient to type it. No, that inconvenience is the whole point.

Step 2: Get the Public Key onto the Server

The easy way:

ssh-copy-id user@server

This copies your public key (~/.ssh/id_ed25519.pub) to the server's ~/.ssh/authorized_keys file and sets correct permissions.

The manual way (if ssh-copy-id isn't available):

cat ~/.ssh/id_ed25519.pub | ssh user@server "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Both methods do the same thing. Pick whichever feels right.

Step 3: Harden the Server Configuration

Logging in with a key is only half the battle. You need to tell the server to reject password auth entirely:

Edit /etc/ssh/sshd_config on the server:

PasswordAuthentication no PubkeyAuthentication yes ChallengeResponseAuthentication no UsePAM no

Then restart SSH:

sudo systemctl restart sshd

Critical: Before you disable password auth, verify you can actually log in with your key. Close your current session, open a new one, and test. If you lock yourself out, you'll need physical access or a console to fix it — and that's a bad day.

Step 4: Manage Your Keys Properly

Most people create one key and use it everywhere. This is convenient until that one key gets compromised, and then you have to rotate it across every server you own.

Better approach: create separate keys for different contexts.

ssh-keygen -t ed25519 -C "work-laptop" ssh-keygen -t ed25519 -C "home-server" ssh-keygen -t ed25519 -C "ci-cd-pipeline"

Then configure ~/.ssh/config to use the right key for the right host:

Host work-* IdentityFile ~/.ssh/id_work_laptop Host home-* IdentityFile ~/.ssh/id_home_server Host ci-* IdentityFile ~/.ssh/id_ci_cd

This keeps your keys compartmentalized. If your CI/CD key leaks, you only rotate that one — not your personal access to every server.

The Mistake Everyone Makes

People forget to set permissions on their private key. If ~/.ssh/id_ed25519 is world-readable, SSH will refuse to use it:

Permissions 0644 for '~/.ssh/id_ed25519' are too open. It is required that your private key files are NOT accessible by others.

Fix it:

chmod 600 ~/.ssh/id_ed25519 chmod 700 ~/.ssh

These permissions are non-negotiable. SSH enforces them strictly.

When Keys Aren't Enough

Key-based auth solves authentication. It doesn't solve authorization. Just because someone has a valid key doesn't mean they should have root on your production database.

Pair key-based auth with:

  • Limited sudo in /etc/sudoers — give users only the commands they need
  • Jump hosts — force all access through a bastion server
  • Agent forwarding caution — forwarding your SSH agent to a remote server means that server can use your keys. Trust the remote, or don't forward.

A Final Thought

SSH key-based authentication isn't complicated. It's just slightly more work than typing a password. That extra work is what makes it secure.

If you're still using passwords in 2026, you're not being careful — you're being lazy. And in security, laziness is the most expensive trait you can have.

Tags: ssh, security, linux, devops, tutorial

More field-tested ops notes and server security guides: jiunotes · ops notes