Ubuntu Server SSH Setup and Configuration Guide

Overview and goals

This guide shows how to install, secure, manage, and troubleshoot OpenSSH on a Linux server. You will learn practical steps to run SSH safely: install the server, configure sshd_config for security and performance, switch to key-based logins, harden access with firewall and tools like fail2ban, and keep keys and backups under control. Each section gives commands and checks you can copy and run.

Prerequisites and system preparation

You need a Linux server with root or sudo access and a separate client machine to test connections. Make sure you have console or out-of-band access (hosting panel, serial console) before making changes that could lock you out.

Check basic info:

• OS and package manager (Ubuntu/Debian use apt; CentOS/RHEL use dnf/yum).
• Current OpenSSH package: on the server run:
  • apt show openssh-server
  • rpm -q openssh-server
• Client OpenSSH version: ssh -V

Keep the system updated before major changes:

• Ubuntu/Debian: sudo apt update && sudo apt upgrade
• RHEL/CentOS: sudo dnf update

Installing and verifying OpenSSH Server

Install OpenSSH server:

• Ubuntu/Debian:
  sudo apt install openssh-server

• RHEL/CentOS:
  sudo dnf install openssh-server

Start and enable the service after install (service name differs by distro):

• Systemd start now: sudo systemctl start sshd (or sudo systemctl start ssh on Debian)
• Enable at boot: sudo systemctl enable sshd

Verify installation and version:

• ssh -V # client version
• sudo sshd -T # prints effective server config (requires root)
• sudo apt show openssh-server (or rpm -qi openssh-server)

Check host keys exist:

• ls -l /etc/ssh/ssh_host_*

If host keys are missing, generate them:

• sudo ssh-keygen -A

Starting, enabling, and checking the SSH service

Manage the service with systemd:

• Start: sudo systemctl start sshd
• Enable: sudo systemctl enable sshd
• Status: sudo systemctl status sshd
• Logs (recent): sudo journalctl -u sshd –no-pager -n 200

On Debian the service is often named ssh. If a command fails, try both ssh and sshd.

Test remote connect locally first:

• From the server: ssh localhost -vvv
  This helps catch config errors before connecting over the network.

Basic SSH client usage and secure connection practices

Connect to a server:

• ssh user@server_ip

Useful client options:

• -p PORT to specify nonstandard port.
• -i /path/to/key to use a private key file.
• -vvv to troubleshoot connection steps.

Use a per-host config in ~/.ssh/config to simplify connections:
Host myserver
HostName server.example.com
User deploy
Port 2222
IdentityFile ~/.ssh/id_ed25519

Keep these practices:

• Use strong, unique accounts (avoid using root for interactive login).
• Use passphrases on private keys.
• Avoid agent forwarding on untrusted machines (ForwardAgent no unless needed).
• Verify host keys on first connect and watch for warnings about changed host keys.

Generating SSH key pairs and deploying public keys

Generate modern keys:

• ed25519 (recommended): ssh-keygen -t ed25519 -a 100 -f ~/.ssh/id_ed25519
• RSA if needed: ssh-keygen -t rsa -b 4096 -o -a 100 -f ~/.ssh/id_rsa

-a sets KDF rounds for stronger passphrase protection.

Deploy the public key to the server:

• Use ssh-copy-id (easy): ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server
• Manual: create ~/.ssh on server with correct permissions, then append public key to ~/.ssh/authorized_keys:
  mkdir -p ~/.ssh
  chmod 700 ~/.ssh
  echo “ssh-ed25519 AAAA… user@host” >> ~/.ssh/authorized_keys
  chmod 600 ~/.ssh/authorized_keys

Permissions are critical: .ssh must be 700 and authorized_keys 600. Otherwise the server will ignore keys.

Test key login from your client (keep an open session until confirmed):

• ssh -i ~/.ssh/id_ed25519 user@server

Configuring sshd_config for security and performance

Edit /etc/ssh/sshd_config with care. After editing, test and reload (see below).

Recommended settings (examples and notes):

• Specify host keys (prefer ed25519 first):
  HostKey /etc/ssh/ssh_host_ed25519_key
  HostKey /etc/ssh/ssh_host_rsa_key

• Protocols: modern OpenSSH uses only protocol 2 by default; no need to set Protocol 2.

• Disable root login:
  PermitRootLogin no

• Limit authentication methods:
  PasswordAuthentication yes # temporary while testing
  ChallengeResponseAuthentication no
  UsePAM yes

• Strengthen timeouts and attempts:
  LoginGraceTime 30s
  MaxAuthTries 3
  MaxSessions 10

• Idle and forwarding:
  ClientAliveInterval 300
  ClientAliveCountMax 2
  AllowTcpForwarding no # set to yes only if you need it
  X11Forwarding no

• Limit users/groups:
  AllowUsers alice deploy
  AllowGroups sshusers

• Subsystem for SFTP-only users:
  Subsystem sftp internal-sftp

• Logging:
  LogLevel VERBOSE

After making changes, validate and reload:

• sudo sshd -t # test config file for syntax errors
• sudo systemctl reload sshd

Always test a new SSH connection from another terminal before closing working sessions.

Disabling password authentication and enforcing key-based login

Plan steps to avoid locking yourself out:

1. Ensure key-based login works and you can connect in a separate session.
2. Keep an open session with sudo/root access for rollback or console access.

Change these settings in /etc/ssh/sshd_config:

• PasswordAuthentication no
• ChallengeResponseAuthentication no
• UsePAM yes # you can keep PAM active for other services
• AuthenticationMethods publickey # or “publickey,keyboard-interactive” for 2FA

Test:

• sudo sshd -t
• sudo systemctl reload sshd
• From client, connect with -i and -vvv to confirm you are permitted.

If you are locked out, use hosting provider console or recover via single-user mode to revert.

Firewall and network considerations (UFW, ports, NAT)

Control SSH access at the host firewall and the network edge.

UFW (Ubuntu) basics:

• Allow default port 22: sudo ufw allow 22/tcp
• Allow custom port: sudo ufw allow 2222/tcp
• To limit brute force: sudo ufw limit 22/tcp
• Enable: sudo ufw enable
• Status: sudo ufw status verbose

iptables/nftables and firewalld users should create equivalent rules.

Changing the SSH port helps reduce casual scans but is not a security fix. If you change ports, update any automated tools and port-forwarding rules.

NAT / Port forwarding:

• For servers behind a home router, forward a public port to the server’s LAN IP and port.
• Keep router firmware updated and only forward the necessary port.
• Consider hosting VPN or jump host instead of opening SSH to the public internet.

Limit access by IP where possible:

• UFW: sudo ufw allow from 203.0.113.5 to any port 22 proto tcp
• Or use firewall rules to restrict networks.

Consider monitoring and alerting for new connections.

Additional hardening (fail2ban, two-factor, chroot/jails)

fail2ban

• Install and enable fail2ban to ban IPs after failed attempts:
  sudo apt install fail2ban
• Basic jail config in /etc/fail2ban/jail.d/sshd.local:
  [sshd]
  enabled = true
  port = 22
  filter = sshd
  logpath = /var/log/auth.log
  maxretry = 5
• Restart: sudo systemctl restart fail2ban

Two-factor authentication (2FA)

• Option 1: TOTP (Google Authenticator / libpam-google-authenticator)
  • sudo apt install libpam-google-authenticator
  • Each user runs: google-authenticator
  • Edit /etc/pam.d/sshd to include: auth required pam_google_authenticator.so nullok
  • In sshd_config: ChallengeResponseAuthentication yes
  • For publickey + TOTP, set AuthenticationMethods publickey,keyboard-interactive
• Option 2: Commercial 2FA (Duo) integrates via PAM and offers enterprise features.

Chroot / SFTP jails

• For SFTP-only users, use internal-sftp and Match blocks:
  Subsystem sftp internal-sftp

  Match Group sftpusers
  ChrootDirectory /home/%u
  ForceCommand internal-sftp
  X11Forwarding no
  AllowTcpForwarding no

• Important: ChrootDirectory must be owned by root and not writable by the jailed user. Provide writable subfolders inside (e.g., /home/joe/uploads).

Container or jail alternatives

• For full shell isolation, consider containers (LXC, Docker) or restricted shells. Chroot alone is tricky; test carefully.

Keep SSH and PAM configurations consistent. Test logins for affected users.

Managing SSH keys, agents, and key rotation

Key storage and protection

• Protect private keys with a passphrase.
• File permissions: chmod 600 ~/.ssh/id_*
• Use hardware tokens (YubiKey, Nitrokey) for high security.

ssh-agent and ssh-add

• Use ssh-agent to cache unlocked keys during a session.
• Add key: ssh-add ~/.ssh/id_ed25519
• Beware of agent forwarding (ForwardAgent) to remote machines — it can expose keys.

Key rotation and best practices

• Rotate keys periodically and when a key may be compromised.
• Use a staged rollout: add new key to authorized_keys, confirm new key works, then remove old key.
• Consider OpenSSH certificate authority (CA) for easier rotation:
  • Create CA key and sign user keys with ssh-keygen -s ca_key -I id -n user -V +52w user_key.pub
  • Configure server to trust CA: append CA public key to /etc/ssh/ssh_known_hosts with cert-authority marker or use TrustedUserCAKeys in sshd_config.
  • Certificates let you set expiration and revoke by removing from server config or using revocation lists.

Revocation

• Maintain a list of revoked keys or certificates. For certs, you can use a revocation list and check with AuthorizedKeysCommand if needed.

Inventory and auditing

• Keep a simple inventory of authorized keys and where they live (per account, per host).
• Remove keys for users who leave or change roles.

Troubleshooting, logs, backups, and recovery

Common checks

• Permissions: ~/.ssh 700, ~/.ssh/authorized_keys 600, /etc/ssh/* owned by root.
• Syntax: sudo sshd -t
• Logs:
  • Debian/Ubuntu: sudo tail -n 200 /var/log/auth.log
  • RHEL/CentOS: sudo tail -n 200 /var/log/secure
  • systemd: sudo journalctl -u sshd –no-pager

Typical error causes

• “Permission denied (publickey)”: wrong permissions, wrong public key, server uses different account, authorized_keys format wrong.
• “Host key verification failed”: known_hosts has a different host key; confirm whether host key changed. If expected, remove the old key in ~/.ssh/known_hosts.

Debugging tools

• Client side: ssh -vvv user@host shows the handshake steps and why auth failed.
• Server side: increase LogLevel to VERBOSE temporarily to get more detail.

Backups

• Backup /etc/ssh and host keys: sudo tar czf ssh-backup-$(date +%F).tgz /etc/ssh
• Store backups securely and restrict access.
• Backup user ~/.ssh/authorized_keys where appropriate.

Recovery tips

• Keep a secondary access method (console, hosting control panel).
• If you lose host keys, run sudo ssh-keygen -A to regenerate; clients will warn about changed keys.
• If you lock out SSH by disabling passwords too soon, revert via console or single-user mode.

Regular maintenance

• Watch logs for repeated failed attempts and adjust fail2ban thresholds.
• Keep OpenSSH updated and review sshd_config after upgrades.
• Periodically audit authorized_keys and user accounts.

Conclusion

SSH gives secure remote access when configured correctly. Follow these practical steps: install and verify OpenSSH, enforce key-based authentication, lock down sshd_config, use host- and network-level controls, add defense-in-depth with fail2ban and 2FA, and manage keys and backups carefully. Test changes from a separate session and keep a recovery path ready. With these practices you reduce risk and keep remote access reliable.