Ssh – `ssh-agent` asks passphrase after it has been added

I'm new to the ssh-agent and encounter what I identify as a "bug".

Situation

• I have a passphrase-protected private key.
• I want to use the ssh-agent so I do not write the passphrase each time I ssh.
• ssh-agent adds the private key (according to ssh-agent -l displaying the private key).
• yet when I try to ssh to the remote server with the private key (thanks to the ~/.ssh/config file), the ssh-agent still asks for my passphrase!

Environment

I'm on fedora Linux 4.5.7-202.fc23.x86_64 #1 SMP Tue Jun 28 18:22:51 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux with the ssh version being OpenSSH_7.2p2, OpenSSL 1.0.2h-fips.
Here are two sequences that do not work using the (what I think) GNOME 3 keyring agent and the ssh-agent.

Here is the ~/.ssh/config:

 IdentitiesOnly yes
[..]
Host root.w.com
    HostName 92.1.2.3
    User user
    Port 22
    IdentityFile /home/user/.ssh/key-rsa.priv

GNOME 3 keyring agent

user@local:~$ ssh server-key
Enter passphrase for key '/home/user/.ssh/key-rsa.priv': 
You have new mail.
Last login: Sat Aug 13 10:41:46 2016 from some.ip.dot.org
user@remote:~$
Connection to <remote-ip> closed.
user@local:~$ echo $SSH_AUTH_SOCK
/run/user/1000/keyring/ssh
user@local:~$ echo $SSH_AGENT_PID

user@local:~$ ssh-add ~/.ssh/key-rsa.priv
Enter passphrase for /home/user/.ssh/key-rsa.priv: 
Identity added: /home/user/.ssh/key-rsa.priv (/home/user/.ssh/key-rsa.priv)
user@local:~$ ssh-add -l
4096 SHA256:aZl81hzUczH+sX+/5+tCJHln11xqta62RbtzLQt5LKE /home/user/.ssh/key-rsa.priv (RSA)
user@local:~$ ssh server-key 
Enter passphrase for key '/home/user/.ssh/key-rsa.priv': 
✘  user@local:~$ 

ssh-agent agent

user@local:~$ eval $(ssh-agent)  
Agent pid 3169
user@local:~$ echo $SSH_AGENT_PID
3169
user@local:~$ echo $SSH_AUTH_SOCK
/tmp/ssh-nqpXUUf2qNpT/agent.3168
user@local:~$ ssh-add -D
All identities removed.
user@local:~$ ssh-add ~/.ssh/key-rsa.priv
Enter passphrase for /home/user/.ssh/key-rsa.priv: 
Identity added: /home/user/.ssh/key-rsa.priv (/home/user/.ssh/key-rsa.priv)
user@local:~$ ssh server-key 
Enter passphrase for key '/home/user/.ssh/key-rsa.priv': 
✘  user@local:~$ 

Question

• What should I do so the ssh-agent do not ask the passphrase?
• Any idea why does these not work?

My issue is similar to this question though I read the answer and the solution did not work for me.