I am working on a Linux embedded distribution and I need to monitor incoming and outcoming connections (if they are still actives etc.) and I am experiencing issues because I noticed that, even if the remote host turn off the connection, I can see the connection as ESTABLISHED through netstat.
I have two different versions of this Linux embedded system: one which uses an old 2.6.34 kernel and it works fine, while the one which is failing uses a kernel 3.18.18.
Looking for more information I suspect I had to set/change TCP timeouts, so I configured them through syctl.conf but doesn't change anything.
The system which uses 2.6.34 kernel doesn't have TCP timeouts configured, so it makes me think that TCP are not the root of my problem.
Do you have an idea about what could be the reason of the fact that the connection remain as ESTABLISHED?
Best Answer
If a host disconnects from the other and you still see its connection as
ESTABLISHED, it's probably related to the fact it's not honoring theTCPprotocol and not closing the connection cleanly.The
netstatoutput is an interpreter of the current state ofTCPconnections. If a client wants to disconnect/close the socket that has previously been open and established, they should notify this to the remote system. This is done sending theFIN requestto the other node (more info here), in this case, the server.It they fail to do so, the client indeed disconnects, but the remote server keeps thinking that the client is still connected and thus keeping their state as
ESTABLISHED, and that's where thetcp_keepalive_timeparameter join the equation. As no further packets will be received, the kernel will wait the specified time to this parameter to time-out the connection and forcibly close it.You can debug this issue using the
tcpdumptool. You could simply listen to the connection from the client host on the server side and check if it fails to send theFINrequest.