MacOS – Cannot connect to SSH server from Mac OSX

There is a lot of conflicting information I've read whenever I look up information on using ssh-agent (passphrase saving/reusing process) under Mac OS X. Most resources seem to suggest that simply issuing ssh-add -K will let you store your passphrase, and will automatically configure OS X to launch ssh-agent automatically and load your stored passphrase.

Note: Running ssh-add -K will only work if you have your private key file in one of the common locations, those locations being limited to: ~/.ssh/id_rsa, ~/.ssh/id_dsa, ~/.ssh/identity. If the file is located anywhere else you should specify that path after the -K in the command above.

The reason you are getting the key file passphrase dialog when connecting to the second (key-less) server is likely because the default configuration of SSH servers is to use public key authentication first, and 'keyboard interactive' authentication second.

Because you have a public key with a standard name/location (~/.ssh/id_rsa), your OpenSSH client helpfully submits the private key in order to allow the server to match it against an allowed authorized_keys file.

There are a small handful of ways to prevent this, the easiest two being to pass a flag on the command line, or add it as a permanent configuration item in your ~/.ssh/config file.

1. When connecting to the secondary/key-less server, you can add -o "PubkeyAuthentication=no" when connecting. Something like ssh -o "PubkeyAuthentication=no" me@devserver2.

2. Open up ~/.ssh/config in your favorite text editor, create it first if you must, and enter the following:

   Host devserver2  
       User me  
       PubkeyAuthentication no
   

Now, if you simply type ssh devserver2 the username and pubkey configuration will be read in and used, and you should be prompted for your password and nothing else.

(Note: Replace devserver2 with the actual hostname of the server. Alternatively, pick a nice hostname, such as devserver2, and add a property between User and PubkeyAuthentication called 'Hostname' and put the name or IP address of the server there. Afterwards, you actually can simply type 'ssh devserver2' and all the configuration properties will work their respective magic.)

OS X actually has (at least) 3 firewalls. Since you've turned off the application firewall (in System Preferences -> Security & Privacy -> Firewall) and checked the Berkeley packet filter (pfctl -sa), I'm guessing it's the old ipfw that's doing the blocking. You can check with sudo ipfw show -- that'll list the active rules, along with counts of how many packets and bytes each one has applied to:

$ sudo ipfw show
01000 19228642 23229993542 allow ip from any to any via lo0
01010        0           0 deny ip from any to 127.0.0.0/8
[etc...]
65534    23505     3467352 deny ip from any to any
65535        0           0 allow ip from any to any

If your listing only shows rule #65535 (the allow rule at the end), my guess is wrong and you have to look elsewhere. If it does show other rules, you probably have a third-party firewall config program installed somewhere (I don't think the Apple-supplied ipfw config software is still there in 10.8); take a look in /Library/StartupItems and /Library/LaunchDaemons for things that might be relevant.