Automatically Added Keys

There are three ways SSH keys are added to ssh-agent in Snow Leopard.

• manually, with ssh-add,
• automatically, by ssh when you supply a key’s passphrase via the GUI prompt, and
• automatically, by ssh-agent when it first starts.

The last two methods are Apple extensions: there are no “automatic” additions with stock OpenSSH. All references to ssh, ssh-agent, and ssh-add below are to Apple’s Snow Leopard versions unless I prefix the program name with the adjective “stock”.

You can disable all of Apple’s keychain-oriented SSH modifications with a (undocumented?) preference setting:

defaults write org.openbsd.openssh KeychainIntegration -bool false

Keys Added Automatically By ssh

(This is the part I missed in the previous version of my original answer since I usually use a “stock” ssh.)

Whenever ssh tries to use a passphrase protected SSH key to authenticate itself to a remote host, it will issue a GUI prompt for the SSH key’s password. The key is also loaded into the agent (if the passphrase is correct) whether or not you mark the “Remember password in my keychain” checkbox.

There are two (undocumented?) ways to prevent ssh from issuing this GUI prompt (and thus adding the SSH key to the ssh-agent):

• A preference setting:

  defaults write org.openbsd.openssh AskPassGUI -bool false
  

• An ssh_config entry (or -o option to ssh) that specifies AskPassGUI no.

  (see keychain_read_passphrase in keychain.c; the oAskPassGUI parameter comes from the AskPassGUI configuration setting)

When AskPassGUI is disabled, ssh will prompt you in the normal way for the key’s password (i.e. through the tty).

You could also avoid automatic adds from ssh by using a “stock” ssh (e.g. OpenSSH compiled by MacPorts, Homebrew “duplicates” from homebrew-alt, or Fink).

Keys Added Automatically By ssh-agent

The keys that ssh-agent automatically adds are those that have their passphrases stored in a keychain. These “remembered keys” are automatically added when a new ssh-agent starts. There is no command line or configuration option (other than KeychainIntegration, described above) to prevent ssh-agent from automatically loading the “remembered keys” (see the call to process_add_from_keychain (defined in keychain.c) from main in ssh-agent.c). If, however, you can arrange to lock the keychains that store your SSH key passphrases, you can click Cancel when ssh-agent asks to unlock the keychain(s) and effectively get ssh-agent to skip adding these “remembered keys” when it first starts.

If there is no ssh-agent running, your first use of ssh will likely trigger launchd to start an ssh-agent which will load all the “remembered keys”. This makes it seem like ssh is loading the keys into the agent, but it is really the agent itself that is loading the keys. It only does this automatically when it first starts.

The -k option of ssh-add provides a manual way to add the “remembered keys” (see add_from_keychain in ssh-add.c which ends up as a message to the agent which calls process_add_from_keychain from process_message in ssh-agent.c).

launchd Configuration

You are right that a system update could overwrite your modification to the file in /System/Library/LaunchAgents/. You should always avoid changing things under /System/; most things can be (re)configured without making changes there. In this case, it looks like you should be able to override the system default launchd job specification on a per-user basis with a file in ~/Library/LaunchAgents/.

From what I can tell¹, entries are loaded in this order²:

1. ~/Library/LaunchAgents/
2. /Library/LaunchAgents/
3. /Network/Library/LaunchAgents/ (not present on most systems)
4. /System/Library/LaunchAgents/

It does not seem to be documented, but only the first job configuration for each Label (e.g. org.openbsd.ssh-agent) will be kept. Any configuration from a later directory with the same Label as a configuration from an earlier directory is effectively skipped.

¹ See NSStartSearchPathEnumeration used in launchctl.c and defined in NSSystemDirectories.h/NSSystemDirectories.c.

² launchd also looks in the LaunchDaemons/ directories next to the various LaunchAgents/ directories for other types of jobs.

1. The dialog for ssh-agent can be circumvented by adding the key in the console/terminal: ssh-add ~/.ssh/id_rsa. You can then paste the password into the terminal. Also, adding the -K option to ssh-add will save it to the keychain as per Oliver Lacans comment.
2. As mattmcmanus said, id_rsa encryption using PKCS seems to be broken on OSX Mavericks ssh-agent. The fast workaround is to decrypt the keyfile, and encrypt it again with the standard ssh procedure (Key-derivation method: MD5...):

mv id_rsa id_rsa.pkcs
openssl rsa -in id_rsa.pkcs -out id_rsa
# enter passphrase to decrypt
chmod 0600 id_rsa
ssh-keygen -f id_rsa -p
# enter passphrase to encrypt again