I've just set up a pfSense router, and am trying to figure out some strange behaviour. It's a fairly simple set up: I have a static IP from the ISP and a single PPPoE WAN interface and a single LAN interface. I've allowed DNS everywhere through the firewall, and browsing the internet generally works. I'm using pfsense 2.1-BETA1 (i386) from March 19th.
Issues (these may be related):
-
Large HTTP downloads will just 'stop' at some point, and never complete. I'm trying to download an ISO at the moment, and it has just given up at about 103MB out of 650MB. Despite multiple retries, downloads larger than 50MB never complete.
-
I'm seeting some strange things in the firewall logs about blocking outbound traffic on port 443:
log entry:
Mar 22 18:25:22 192.168.0.1 pf: 00:00:00.818527 rule 4/0(match): block out on pppoe0: (tos 0x0, ttl 63, id 3535, offset 0, flags [DF], proto TCP (6), length 893)
Mar 22 18:25:22 192.168.0.1 pf: <publicip>.44395 > 173.194.78.103.443: Flags [FP.], seq 2278533959:2278534812, ack 270462703, win 262, length 853
and
Mar 22 18:32:10 192.168.0.1 pf: 00:00:22.972286 rule 3/0(match): block in on pppoe0: (tos 0x0, ttl 57, id 39991, offset 0, flags [DF], proto TCP (6), length 84)
Mar 22 18:32:10 192.168.0.1 pf: 173.194.78.103.443 > <publicip>.3684: Flags [FP.], cksum 0x8cdd (correct), seq 1848167695:1848167739, ack 810363008, win 501, length 44
The first appears to be a packet from me to a Google IP address bound for port 443. The second appears to be a packet from the same IP, perhaps a response to a request. Why would this be blocked? In a typical NAT scenario, I would expect outbound packets to be permitted and established/related traffic to be permitted back in.
If this type of traffic is blocked, why can I otherwise browse the web? Why isn't it broken everywhere?
(edit: I'm starting to suspect this may be an MTU problem…)
Best Answer
It's not a direct answer, but I'm seeing almost exactly the same thing on a newer (5.3 release candidate) OpenBSD system. Many bursts of outbound packets getting blocked; many with a destination at Google and/or port 443; many with F(fin) and P(push) set in the options. These are coming from both Windows and OS X-based systems, so it's not just an OS-level bug in the clients.
Unlike your situation, as far as I know the clients are communicating just fine with the Net at large - and also, as far as I know, it's only me who is seeing the underlying failures, by perusing pflog.
I'll be fascinated for any answers or even speculation on what these might be.
PS: I'm on a PPPoE link with max-mss clamped to 1448 to get this all to work; not sure if that's similar to your setup or not, but as you mention MTU issues...
*******EDIT:********* See: http://doc.pfsense.org/index.php/Logs_show_%22blocked%22_for_traffic_from_a_legitimate_connection,_why%3F or http://doc.m0n0.ch/handbook/faq-legit-traffic-dropped.html
Apparently this is perfectly normal behavior involving duplicate FIN packets. I'm just going to add a rule that catches these so they don't clutter up the log.