Ssh – Permit root to login via ssh only with key-based authentication

As mentioned in the comment, you're using an encrypted home directory, and are likely using pam_mount to mount it.
pam_mount mounts the partition using the password acquired during login. Since you're trying to log in via ssh public keys there are 2 issues:

1. There is no password being sent during public key authentication, so it can't mount your home directory using it.
2. When using pam_mount, your home directory is mounted after login, but sshd needs to get your authorized_keys file before login, and thus it's not mounted.

Either of these issues is enough to prevent it from working.

The only solution is to get your public keys out of the home directory. This is actually rather simple.

First copy the authorized_keys file out of the home directory:

cp -a /home/$USER/.ssh/authorized_keys /home/$USER-authorized_keys

Then tell sshd to use that file by adding the following to /etc/ssh/sshd_config (replace existing entry if present):

AuthorizedKeysFile .ssh/authorized_keys /home/%u-authorized_keys

And bounce sshd.

Note however that this will not mount your home directory. Your home directory still needs your password to decrypt. Depending on how you have pam_mount configured, it might prompt you for your password, or it might just drop you into a shell with your home unmounted.

First, the .ssh directory should have 700 permissions and the authorized_keys file should have 600.

chmod 700 .ssh
chmod 600 .ssh/authorized_keys

In case you created the files with say root for userB then also do:

chown -R userb:userb .ssh

If the problem still persist, then post the output from your ssh log file in your question and I'll update my answer.

For Debian:

less /var/log/auth

For Redhat:

less /var/log/secure