SSH Authentication – Why Still Asked for Password with Private Key?

You need your SSH public key and you will need your ssh private key. Keys can be generated with ssh-keygen. The private key must be kept on Server 1 and the public key must be stored on Server 2.

This is completly described in the manpage of openssh, so I will quote a lot of it. You should read the section 'Authentication'. Also the openSSH manual should be really helpful: http://www.openssh.org/manual.html

Please be careful with ssh because this affects the security of your server.

From man ssh:

 ~/.ssh/identity
 ~/.ssh/id_dsa
 ~/.ssh/id_rsa
     Contains the private key for authentication.  These files contain
     sensitive data and should be readable by the user but not acces-
     sible by others (read/write/execute).  ssh will simply ignore a
     private key file if it is accessible by others.  It is possible
     to specify a passphrase when generating the key which will be
     used to encrypt the sensitive part of this file using 3DES.

 ~/.ssh/identity.pub
 ~/.ssh/id_dsa.pub
 ~/.ssh/id_rsa.pub
     Contains the public key for authentication.  These files are not
     sensitive and can (but need not) be readable by anyone.

This means you can store your private key in your home directory in .ssh. Another possibility is to tell ssh via the -i parameter switch to use a special identity file. Also from man ssh:

 -i identity_file
     Selects a file from which the identity (private key) for RSA or
     DSA authentication is read.  The default is ~/.ssh/identity for
     protocol version 1, and ~/.ssh/id_rsa and ~/.ssh/id_dsa for pro-
     tocol version 2.  Identity files may also be specified on a per-
     host basis in the configuration file.  It is possible to have
     multiple -i options (and multiple identities specified in config-
     uration files).

This is for the private key. Now you need to introduce your public key on Server 2. Again a quote from man ssh:

  ~/.ssh/authorized_keys
         Lists the public keys (RSA/DSA) that can be used for logging in
         as this user.  The format of this file is described in the
         sshd(8) manual page.  This file is not highly sensitive, but the
         recommended permissions are read/write for the user, and not
         accessible by others.

The easiest way to achive that is to copy the file to Server 2 and append it to the authorized_keys file:

scp -p your_pub_key.pub user@host:
ssh user@host
host$ cat id_dsa.pub >> ~/.ssh/authorized_keys

Authorisation via public key must be allowed for the ssh daemon, see man ssh_config. Usually this can be done by adding the following statement to the config file:

PubkeyAuthentication yes

If you can use SSH agent forwarding, there actually is a way: pam_ssh_agent_auth.so (source here) is a PAM module that can do what you ask. It's available in Debian and Ubuntu as package libpam-ssh-agent-auth and as CentOS package pam_ssh_agent_auth.

# Debian/Ubuntu:
apt update; apt install libpam-ssh-agent-auth
# CentoOS
yum install pam_ssh_agent_auth

Security Considerations

You should evaluate the risks of using SSH agent forwarding, as the developer says:

There are caveats of course, ssh-agent forwarding has it’s own security risks which must be carefully considered for your environment. In cases where there are not untrustworthy intermediate servers, and you wish to retain traceability, accountability, and required authentication for privileged command invocation, the benefits should outweigh the risks.

If you make sure your KeeAgent has the option Always require confirmation when client program requests to use key set, this even offers you a degree of protection against someone else with root access on the remote host: if you get a SSH key request confirmation dialog with no obvious reason, you'll know that someone is trying to abuse your SSH agent connection.

If you also make sure you'll always lock your KeePass and/or workstation screen when you step away from it, I think this should offer pretty good security; it's certainly more secure than using NOPASSWD in sudoers. It is also better than allowing root logins with ssh keys only and adding everyone who is allowed to log in as root to root's authorized_keys file because it maintains the advantages of sudo.

Usage

To use it, you basically add this as the first auth line in /etc/pam.d/sudo:

auth    sufficient      pam_ssh_agent_auth.so file=/etc/security/authorized_keys

A simple sed command to do this (adds it to the second line, since the first is a comment):

sed -i '2 i\auth    sufficient      pam_ssh_agent_auth.so file=/etc/security/authorized_keys' /etc/pam.d/sudo

Then add the public SSH keys of users that should be authorized to use SSH-authenticated sudo to /etc/security/authorized_keys in the usual single-line OpenSSH compatible format.

Then configure sudoers to preserve the environment variable SSH_AUTH_SOCK by editing the sudoers file (use visudo). Add this line to the section with the other Defaults.

Defaults env_keep += "SSH_AUTH_SOCK"

Then, you need to make sure that your ssh client allows agent forwarding. In PuTTY you need to check this:

While testing this, don't forget to terminate your sudo sessions with sudo -k.