First, the .ssh directory should have 700 permissions and the authorized_keys file should have 600.
chmod 700 .ssh
chmod 600 .ssh/authorized_keys
In case you created the files with say root for userB then also do:
chown -R userb:userb .ssh
If the problem still persist, then post the output from your ssh log file in your question and I'll update my answer.
For Debian:
less /var/log/auth
For Redhat:
less /var/log/secure
Here is the solution from the link I posted in my comment. This comes from here, which references this superuser post.
Create .ssh folder in /home for the keys to be stored
sudo mkdir /home/.ssh
Move existing authorized_keys file into .ssh dir as username
sudo mv ~/.ssh/authorized_keys /home/.ssh/username
Create symbolic link to authorized_keys file in user .ssh dir
ln -s /home/.ssh/username ~/.ssh/authorized_keys
Update sshd_config file to set the new path for the authorized_keys file
sudo vim /etc/ssh/sshd_config
Change the AuthorizedKeysFile line to:
AuthorizedKeysFile /home/.ssh/%u
Reboot the computer
sudo shutdown -r now
Login to your server and you should be presented with a minimal un-decrypted home directory... You will need to create and edit a .profile file in there to get ecryptfs to mount your home directory.
sudo vim ~/.profile
Add these lines:
ecryptfs-mount-private
cd /home/username
Log out/Restart, and go back in again. You should be prompted for your password after SSH key auth, and then be presented with your decrypted home directory.
You should now be able to login using SSH keys every time, no matter if your home dir is decrypted or not.
Best Answer
In the ssh_config file, you can can change the location of where it looks for your private key. You could probably do something like make a new folder at
/etc/ssh/keys/and put yourid_rsaprivate key file in there and then change theIdentityFileoption inssh_configto look in the new location. In doing so you'll want to take certain measures to secure your private key.This is assuming you're the only user of the computer. If not, you can make folders like
/etc/ssh/keys/john/and/etc/ssh/keys/dogbert/and then in theIdentityFileoption put/etc/ssh/keys/%u/id_rsa