The fastest way to get a Linux box compromised is to stand it up with a public IP, password login, and root SSH enabled, then walk away. Bots find it within minutes. The good news is that SSH hardening is mostly a one-time checklist, and getting it right removes the single most common path attackers use to land on a server. Here are ten concrete steps, in the order you should do them.
Work through these on a non-production host first, and keep a second terminal session open while you change SSH config. If you lock yourself out, that open session is your way back in.
Step 1: Use keys, not passwords
Password authentication is brute-forceable. Key-based auth is not, in any practical sense. Generate a modern key on your workstation:
ssh-keygen -t ed25519 -C "you@workstation"
Copy the public key to the server with ssh-copy-id user@host, confirm you can log in with the key, and only then move on to disabling passwords.
Step 2: Disable password and root login
In /etc/ssh/sshd_config, set the following:
PasswordAuthentication noPermitRootLogin noChallengeResponseAuthentication no
Log in as a normal user and escalate with sudo instead of logging in as root directly. This gives you an audit trail and removes the most-targeted account name.
Step 3: Restrict who can log in
Do not let every system account be a login target. Add an explicit allowlist:
AllowUsers deploy admin
Now only those two accounts can authenticate over SSH, even if another account exists and has a key.
Step 4: Change the default assumptions, not just the port
Moving SSH off port 22 cuts down log noise from automated scanners, but treat it as tidiness, not security. A port scan finds your new port in seconds. Do it if you like quieter logs, but never rely on it as a control. Real protection comes from the other steps here.
Step 5: Put SSH behind a firewall
Limit where connections can even originate. With ufw:
ufw allow from 203.0.113.0/24 to any port 22 proto tcp
If your team connects from a known office range or a VPN, allowlist only that. An attacker who cannot reach the port cannot attack it.
Step 6: Add rate limiting with Fail2ban
Even with keys only, you want to throttle probing. Fail2ban watches your auth log and bans IPs that misbehave. Install it, enable the sshd jail, and set a sane bantime. It will not stop a targeted attacker, but it cleans up the constant background noise and slows credential stuffing to a crawl.
Step 7: Tighten the crypto
Disable weak ciphers, MACs, and key exchange algorithms. The Mozilla OpenSSH guidelines are a maintained reference for a modern, safe set. Pin strong algorithms in sshd_config so a downgrade is not possible, and remove anything legacy that you do not explicitly need.
Step 8: Set idle and login timeouts
Drop idle sessions and slow-walking login attempts:
ClientAliveInterval 300andClientAliveCountMax 2to close idle sessionsLoginGraceTime 20to cut off connections that stall before authenticatingMaxAuthTries 3to limit guesses per connection
Step 9: Log, then actually read the logs
Hardening without monitoring is half a job. Ship your auth logs somewhere central, and alert on repeated failures, logins from new geographies, or any successful root login (which should now be impossible). This is exactly the kind of signal an AI triage agent can watch for you, a pattern we covered in our piece on agentic incident response.
Step 10: Validate and re-test
After every change, run sshd -t to check the config syntax before restarting the service. Then reconnect from a fresh terminal to confirm you are not locked out. Re-run a quick audit periodically, because a server hardened in January drifts by June as people add keys and tweak settings.
Putting it together
Keys only, no root, an allowlist of users and source IPs, rate limiting, modern crypto, and real log monitoring. That stack defeats the overwhelming majority of attacks against SSH, which are opportunistic and automated. If you want to build the Linux fundamentals these steps assume, our Linux CLI Coach and CompTIA Security+ Cert Coach walk through them hands-on, and the full course library covers the rest of the hardening picture. For the canonical reference, keep the OpenSSH manual and the CIS Benchmarks bookmarked.
Frequently asked questions
Is changing the SSH port enough to secure a server?
No. Moving off port 22 reduces automated scan noise in your logs, but any attacker can find the new port with a quick scan. Use it for tidiness, and rely on key-only auth, firewall rules, and rate limiting for actual security.
Should I disable root SSH login even on my own server?
Yes. Logging in as a normal user and escalating with sudo gives you a per-user audit trail and removes the most heavily targeted account name. It is one of the highest-value, lowest-effort changes you can make.
Do I still need Fail2ban if I only allow key-based login?
It is not strictly required, since keys are not brute-forceable, but it is worth running. Fail2ban cuts the constant background probing, keeps your logs readable, and adds a layer against any future misconfiguration that accidentally re-enables passwords.
