sudo ufw disable
followed by sudo ufw enable
kicks me out of SSH
DMESG reports
[UFW BLOCK] IN=eth0 OUT= MAC=30:........ SRC=192.168.1.me DST=192.168.1.server LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=15776 DF PROTO=TCP SPT=55640 DPT=22 WINDOW=253 RES=0x00 ACK URGP=0
I can log back in without having to change rules via the console (UFW still enabled).
This started after upgrading Xenial (16.04) from kernel 4.4 to 4.15 (HWE). Upgrading to 18.04.1 did not solve the issue.
Versions:
- iptables v1.6.1
- ufw 0.35
- 4.15.0-29-generic #31-Ubuntu
- Ubuntu 18.04.1 LTS
UFW status verbose is (some rules were omitted, but they are all ALLOW)
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
22 ALLOW IN Anywhere
22 (v6) ALLOW IN Anywhere (v6)
Why is this happening, or at least, how to revert to the expected behavior?
I looked at this answer, and I am not sure it applies, but here's /etc/ufw/before.rules
#
# rules.before
#
# Rules that should be run before the ufw command line added rules. Custom
# rules should be added to one of these chains:
# ufw-before-input
# ufw-before-output
# ufw-before-forward
#
# Don't delete these required lines, otherwise there will be errors
*filter
:ufw-before-input - [0:0]
:ufw-before-output - [0:0]
:ufw-before-forward - [0:0]
:ufw-not-local - [0:0]
# End required lines
# allow all on loopback
-A ufw-before-input -i lo -j ACCEPT
-A ufw-before-output -o lo -j ACCEPT
# quickly process packets for which we already have a connection
-A ufw-before-input -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-output -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A ufw-before-forward -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# drop INVALID packets (logs these in loglevel medium and higher)
-A ufw-before-input -m conntrack --ctstate INVALID -j ufw-logging-deny
-A ufw-before-input -m conntrack --ctstate INVALID -j DROP
# ok icmp codes for INPUT
-A ufw-before-input -p icmp --icmp-type destination-unreachable -j ACCEPT
-A ufw-before-input -p icmp --icmp-type source-quench -j ACCEPT
-A ufw-before-input -p icmp --icmp-type time-exceeded -j ACCEPT
-A ufw-before-input -p icmp --icmp-type parameter-problem -j ACCEPT
-A ufw-before-input -p icmp --icmp-type echo-request -j ACCEPT
# ok icmp code for FORWARD
-A ufw-before-forward -p icmp --icmp-type destination-unreachable -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type source-quench -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type time-exceeded -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type parameter-problem -j ACCEPT
-A ufw-before-forward -p icmp --icmp-type echo-request -j ACCEPT
# allow dhcp client to work
-A ufw-before-input -p udp --sport 67 --dport 68 -j ACCEPT
#
# ufw-not-local
#
-A ufw-before-input -j ufw-not-local
# if LOCAL, RETURN
-A ufw-not-local -m addrtype --dst-type LOCAL -j RETURN
# if MULTICAST, RETURN
-A ufw-not-local -m addrtype --dst-type MULTICAST -j RETURN
# if BROADCAST, RETURN
-A ufw-not-local -m addrtype --dst-type BROADCAST -j RETURN
# all other non-local packets are dropped
-A ufw-not-local -m limit --limit 3/min --limit-burst 10 -j ufw-logging-deny
-A ufw-not-local -j DROP
# allow MULTICAST mDNS for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 224.0.0.251 --dport 5353 -j ACCEPT
# allow MULTICAST UPnP for service discovery (be sure the MULTICAST line above
# is uncommented)
-A ufw-before-input -p udp -d 239.255.255.250 --dport 1900 -j ACCEPT
# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
PS: I didn't expect this to "fix" the issue but just for reference I changed the port SSHD listens on (and the corresponding rule) and the problem persists.
Best Answer
Background, and bounds for the issue:
What is going on?
The
sudo ufw allow in port 22
results in the following iptables rules segment:Upon
sudo ufw disable
followed bysudo ufw enable
, and even though the ssh connection itself remains fine, the resulting iptables rule set seems to have forgotten the association with that particular connection and therefore classifies any incoming packets as invalid. Somehow the connection tracking table has become confused and the packet is not even considered NEW, but with incorrect flags, nor is it considered part of the existing connection.Consider a very basic iptables equivalent of what
ufw
is doing. Two scripts, one for clearing the rule set and one for creating it:And:
Resulting in these packets counts after a clear/load cycle with an ssh session that was started after a load cycle:
Notice the 35 invalid packets as I typed on the crippled ssh session terminal, and before PuTTY terminated.
Why did this stop working, it used to work?
Because this is 100% repeatable, a kernel bisection was relatively easy, just time consuming. The results were:
Link to the entire commit.
How to revert to the expected behavior?
After disabling ufw or clearing the iptables rules set, create a new SSH session. It will survive a subsequent ufw enable, but might be subject to a random drop out at some point.
This issue will be taken upstream at some point, via the related e-mail list.
EDIT: upstream e-mail thread (contains a work around). Workaround copied here:
EDIT 2: upstream proposed patch , which I have tested and reported back.
EDIT 3: 2018.11.06: This has stalled upstream, and I haven't had time to pester them. I'll try to get back to it soon.
EDIT 4: 2019.03.17: I can not reliably reproduce this issue with kernel 5.0.