No prompt for ssh passphrase

sshterminal

I have two ssh keys, one for git (gitlab) and one for a server.
My ~/.ssh/config is :

Host private
  HostName x.x.x.x
  User username
  IdentityFile ~/.ssh/server

Host git.example.com
  IdentityFile ~/.ssh/gitlab

However when I try ssh username@x.x.x.x or git commands I just have a message Permission denied (public key). I have to do ssh-add ~/.ssh/server and ssh-add ~/.ssh/git in order to use the keys.

How can I have a prompt to enter the passphrase the first time the key is used instead of using ssh-add ?
Ideally I'd prefer to have to enter my passphrase after every reboot instead of having the passphrase saved.

I am on macOS Mojave 10.14.6 and I use iTerm2 with oh-my-zsh.

EDIT :
After the excellent answer from wisbucky I also tried :

Host private
  HostName x.x.x.x
  User username
  IdentityFile ~/.ssh/server
  UseKeychain yes
  AddKeysToAgent yes

Host git.example.com
  IdentityFile ~/.ssh/gitlab
  UseKeychain yes
  AddKeysToAgent yes

and

Host *
  UseKeychain yes
  AddKeysToAgent yes

for the ~/.ssh/config file but without results…

Best Answer

In the ~/.ssh/config, add these lines:

Host *
    UseKeychain yes
    # automatically add keys to keychain
    AddKeysToAgent yes

UseKeychain yes will use any saved ssh keys in the Mac Keychain.

AddKeysToAgent yes will automatically save ssh keys in the Mac Keychain after the first time you enter the passphrase. If you don't add this, you can use ssh-add -K to manually add keys to the keychain also.

Note: These options were added since macOS 10.12.2

https://developer.apple.com/library/archive/technotes/tn2449/_index.html

Related Question