"Connection reset by peer" means the TCP stream was abnormally closed from the other end. I think the most likely explanations are that the remote server process handling the connection has crashed, or else some network device (like a stateful firewall or load balancer) has decided to interfere with the connection.
You need to debug this on the remote server if you can. sshd logs through syslog, and on a typical Unix system the log entries will be in one of the files in the /var/log directory. If you're lucky, sshd will be logging something every time it drops your session.
If you have root access on the server, you can run a debugging instance of sshd. Become root and then run:
/path/to/sshd -ddd -p 1022
This will run an instance of the SSH server which will listen on port 1022, accept one connection, and print debugging information to your terminal. Run your client as usual, except specify port 1022 as the port:
ssh -p 1022 user@host
The debugging information printed by the server will hopefully make it clear what is happening.
Edit: The server output indicates that the server isn't crashing or deliberately closing the TCP connection. Something else is causing it to close. I would take a look at any security software installed on the server which might monitor TCP sessions, as well as any firewalls, load balancers, or similar network hardware which might be part of the local network.
Unfortunately I can't comment yet.
What comes to my mind on the client:
Have you had a look at top to see the CPU load? Maybe the SSH process uses up CPU for encryption.
Have you had a look at ssh -v[vv] and inspected for some strangeness? Maybe server and client agree on a very secure cipher or MAC. Have a look for
debug2: ciphers ctos: arcfour
debug2: ciphers stoc: arcfour
[...]
debug1: kex: server->client cipher: arcfour MAC: hmac-sha2-256-etm@openssh.com compression: zlib@openssh.com
debug1: kex: client->server cipher: arcfour MAC: hmac-sha2-256-etm@openssh.com compression: zlib@openssh.com
(where arcfour is really one of the weakest, lowest-CPU algorithms. I am connecting via an SSH proxy). Also, look for rekeying messages.
Also, compression might be an issue. However, ssh does not seem to be too explicit about the compression level here.
debug2: compression ctos: zlib@openssh.com,zlib,none
debug2: compression stoc: zlib@openssh.com,zlib,none
You didn't say explicitly, how 'remote' your remote servers are. In the same network, in a different network on the same campus, via the internet?
If via the internet, maybe your corporate firewall is deep-inspecting SSH traffic on port 22. Maybe a port change could help, if you can control the SSH servers /etc/ssh/sshd_config file.
If via the internet, is SSH the 'only' tunnel you use, or are you using SSH via an additional VPN?
If on the same campus, also firewalling, routing, inspecting issues come to my mind. I once was blaming my internet provider for problems with my internet; it was damn slow. Went on for days. Until I found out that I enabled debugging on my Cisco router, which ate up resources there.
If local or remote, are you connecting directly or via an SSH proxy, gateway or jump host? In that case, like with the VPN, you have double encryption.
Best Answer
Fixed it! The problem was all my external connections not on port 80 were being blocked.