As I was trying in vain to fix a faulty ethernet controller here, one thing I tried was running tcpdump on the machine.
I found it interesting that tcpdump was able to detect that some of the ICMP packets the ping application thought it was sending were not actually going out on the wire, even though it was running on the same machine. I have reproduced those tcpdump results here:
14:25:01.162331 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 1, length 64
14:25:02.168630 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 2, length 64
14:25:02.228192 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 2, length 64
14:25:07.236359 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 3, length 64
14:25:07.259431 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 3, length 64
14:25:31.307707 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 9, length 64
14:25:32.316628 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 10, length 64
14:25:33.324623 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 11, length 64
14:25:33.349896 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 11, length 64
14:25:43.368625 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 17, length 64
14:25:43.394590 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 17, length 64
14:26:18.518391 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 30, length 64
14:26:18.537866 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 30, length 64
14:26:19.519554 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 31, length 64
14:26:20.518588 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 32, length 64
14:26:21.518559 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 33, length 64
14:26:21.538623 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 33, length 64
14:26:37.573641 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 35, length 64
14:26:38.580648 IP debian.local > 74.125.224.80: ICMP echo request, id 2334, seq 36, length 64
14:26:38.602195 IP 74.125.224.80 > debian.local: ICMP echo reply, id 2334, seq 36, length 64
Notice how the seq number jumps several times… that indicates packets the ping application generates that are not actually leaving the box.
Which brings me to my question: how was tcpdump able to detect that the ICMP packets weren't actually going out? Is it able to somehow directly monitor what is on the wire?
If it does accomplish this, I assume it is by interfacing to some part of the kernel, which in turn interfaces to some hardware that is a standard part of a network controller.
Even so, that's pretty cool! If that is not actually how tcpdump functions, can someone explain to me how it detected the missing packets in software?
Best Answer
Yes. By putting network interfaces into promiscuous mode, tcpdump is able to see exactly what is going out (and in) the network interface.
tcpdump operates at layer2 +. it can be used to look at Ethernet, FDDI, PPP & SLIP, Token Ring, and any other protocol supported by libpcap, which does all of tcpdump's heavy lifting.
Have a look at the pcap_datalink() section of the pcap man page for a complete list of the layer 2 protocols that tcpdump (via libpcap) can analyze.
A read of the tcpdump man page will give you a good understanding of how exactly, tcpdump and libpcap interface with the kernel and network interfaces to be able to read the raw data link layer frames.