Automatically Added Keys

There are three ways SSH keys are added to ssh-agent in Snow Leopard.

• manually, with ssh-add,
• automatically, by ssh when you supply a key’s passphrase via the GUI prompt, and
• automatically, by ssh-agent when it first starts.

The last two methods are Apple extensions: there are no “automatic” additions with stock OpenSSH. All references to ssh, ssh-agent, and ssh-add below are to Apple’s Snow Leopard versions unless I prefix the program name with the adjective “stock”.

You can disable all of Apple’s keychain-oriented SSH modifications with a (undocumented?) preference setting:

defaults write org.openbsd.openssh KeychainIntegration -bool false

Keys Added Automatically By ssh

(This is the part I missed in the previous version of my original answer since I usually use a “stock” ssh.)

Whenever ssh tries to use a passphrase protected SSH key to authenticate itself to a remote host, it will issue a GUI prompt for the SSH key’s password. The key is also loaded into the agent (if the passphrase is correct) whether or not you mark the “Remember password in my keychain” checkbox.

There are two (undocumented?) ways to prevent ssh from issuing this GUI prompt (and thus adding the SSH key to the ssh-agent):

• A preference setting:

  defaults write org.openbsd.openssh AskPassGUI -bool false
  

• An ssh_config entry (or -o option to ssh) that specifies AskPassGUI no.

  (see keychain_read_passphrase in keychain.c; the oAskPassGUI parameter comes from the AskPassGUI configuration setting)

When AskPassGUI is disabled, ssh will prompt you in the normal way for the key’s password (i.e. through the tty).

You could also avoid automatic adds from ssh by using a “stock” ssh (e.g. OpenSSH compiled by MacPorts, Homebrew “duplicates” from homebrew-alt, or Fink).

Keys Added Automatically By ssh-agent

The keys that ssh-agent automatically adds are those that have their passphrases stored in a keychain. These “remembered keys” are automatically added when a new ssh-agent starts. There is no command line or configuration option (other than KeychainIntegration, described above) to prevent ssh-agent from automatically loading the “remembered keys” (see the call to process_add_from_keychain (defined in keychain.c) from main in ssh-agent.c). If, however, you can arrange to lock the keychains that store your SSH key passphrases, you can click Cancel when ssh-agent asks to unlock the keychain(s) and effectively get ssh-agent to skip adding these “remembered keys” when it first starts.

If there is no ssh-agent running, your first use of ssh will likely trigger launchd to start an ssh-agent which will load all the “remembered keys”. This makes it seem like ssh is loading the keys into the agent, but it is really the agent itself that is loading the keys. It only does this automatically when it first starts.

The -k option of ssh-add provides a manual way to add the “remembered keys” (see add_from_keychain in ssh-add.c which ends up as a message to the agent which calls process_add_from_keychain from process_message in ssh-agent.c).

launchd Configuration

You are right that a system update could overwrite your modification to the file in /System/Library/LaunchAgents/. You should always avoid changing things under /System/; most things can be (re)configured without making changes there. In this case, it looks like you should be able to override the system default launchd job specification on a per-user basis with a file in ~/Library/LaunchAgents/.

From what I can tell¹, entries are loaded in this order²:

1. ~/Library/LaunchAgents/
2. /Library/LaunchAgents/
3. /Network/Library/LaunchAgents/ (not present on most systems)
4. /System/Library/LaunchAgents/

It does not seem to be documented, but only the first job configuration for each Label (e.g. org.openbsd.ssh-agent) will be kept. Any configuration from a later directory with the same Label as a configuration from an earlier directory is effectively skipped.

¹ See NSStartSearchPathEnumeration used in launchctl.c and defined in NSSystemDirectories.h/NSSystemDirectories.c.

² launchd also looks in the LaunchDaemons/ directories next to the various LaunchAgents/ directories for other types of jobs.

Ask your administrator to look at the sshd log. Or more likely you will need to ask them to run a second instance of sshd on a secondary port with debug turned on, doing this should tell you what the key error is.

sshd is quite a bit tricky when it comes to this, for good reason. On the server you are ssh'ing to, the administrator will have to run a command such as /usr/sbin/sshd -d -p 2222

When running sshd in debug mode, it will handle the first connection, spew out lots (and I do mean lots of output), and then terminate the server. Issuing -p means run the server on port 2222, which is necessary because it can't take over the default port of 22 where the server is normally running.

You will then have to run ssh -p 2222 70.32.90.120 (you don't need -v in this instance). The server's logs should then give you the exact reason that your private key is being rejected from the server. Note that this exchange may need to occur multiple times, because ssh is very stringent when it comes to security, and you may find multiple failures in the process.