A program injects packets on a Linux TAP interface (these packets are coming from a virtual machine). Specifically, these are DHCP requests (so they're UDP). I can see the packets with tcpdump but not with iptables, and they don't reach the local DHCP server either. Why not, and how do I fix that?
Update: I tried injecting IP packets directed to the address of the tap0 interface. I see the ARP requests coming in from the VM in tcpdump -i tap0, but the network layer does not reply. If I send ARP requests to the VM, it sees them and replies to the host (and the replies show up in tcpdump but are otherwise lost).
Another observation: ifconfig tap0 shows that the TX dropped packet count is incremented for each packet that's injected onto the host. Why TX?
# ifconfig tap0
…
TX packets:0 errors:0 dropped:958 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
The long story: On a Linux host (running Ubuntu 10.04), I'm running a virtual machine which amongst other things emulates an Ethernet card. It does so by communicating with a helper program that's in charge of injecting and capturing Ethernet packets onto the hosts's network stack. The virtual machine is an ARM chip emulator, and the helper program is called nicserver; all I know about it is what is found in the ARM documentation.
I want to establish an Ethernet link between the VM and the host, and above that I want an IP link. The VM obtains its IP address over DHCP. I do not want any communication between the VM and the rest of the world, only with the host, so I created a virtual network interface tap0 with
tunctl -u gilles
ifconfig tap0 192.168.56.1 netmask 255.255.255.0 up
nicserver -p 7801 -a tap0 &
Now I boot the VM, and I can see it's sending DHCP requests with tcpdump -n -i tap0 -vv (the DHCP client doesn't time out, I'm just showing one sample request here):
tcpdump: listening on tap0, link-type EN10MB (Ethernet), capture size 96 bytes
18:29:23.941574 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 576)
0.0.0.0.68 > 255.255.255.255.67: [no cksum] BOOTP/DHCP, Request from 02:52:56:47:50:03, length 548, xid 0x238a7979, secs 46, Flags [none] (0x0000)
Client-Ethernet-Address 02:52:56:47:50:03 [|bootp]
I've set up Dnsmasq on the host to serve the requests, but it's not seeing any incoming request. The Dnsmasq server doesn't even see the incoming requests (I straced it). So I tried observing the packets with Iptables. (All the filter/INPUT rules are shown; there are no mangle or nat rules).
Chain INPUT (policy ACCEPT 2366K packets, 5334M bytes)
pkts bytes target prot opt in out source destination
119 39176 LOG udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 LOG flags 4 level 4 prefix `[DHCP request] '
119 39176 DROP udp -- eth1 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67
2 490 LOG udp -- tap0 * 0.0.0.0/0 0.0.0.0/0 LOG flags 4 level 4 prefix `[in=tap0] '
26 6370 ACCEPT udp -- tap0 * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- tap0 * 0.0.0.0/0 0.0.0.0/0
3864 457K ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0
All these incoming DHCP requests are on eth1 (and I'm careful not to ignore these so as not to anger my colleagues and my network admin). Those UDP packets on tap0 come from the local Samba server. The DHCP request packets that I see with tcpdump do not appear to go through the packet filter!
Why do I see incoming broadcast packets on tap0 with tcpdump but not with iptables (nor with programs listening on the machine)? And what do I need to fix so that these packets are seen, as they would be if they were coming on an Ethernet interface?
Best Answer
Here is further guesswork. Hope this is helpful, but it may as well be embarassingly wrong.
tap0 has two ends, the kernel network stack end and the program interface. It seems to me if you supply 'nicserver' with tap0, it won't attach to it the way it is intended with tap devices, using the program interface. Instead, nicserver will simply write to it from the network stack end, and with no application reading from the program interface end, you will end up overflowing the device queue. This explains the dropped packets. Also, no packets will be delivered, which might explain the iptables result.
I guess if you let tcpdump capture on tap0 it actually attaches to the program interface end of tap0 and, voila, you see the packets. I searched the internet but found no source to confirm such behaviour. To falsify this theory, capture tap0 and see how it affects packet drops and the iptables log.
Finally, an advice that addresses your original problem: what about using the loopback device, instead? Like so: