Linux Command Line Tools for Analyzing OS + NIC TCP Latency

I'm hoping to find an existing Linux tool for measuring latency:

I'm trying to diagnose how much latency my OS and NIC card are adding to TCP latency in Linux. I looked at the following tools but all (at least at the surface level) measured bandwidth (Mbps/sec) rather than latency (nanoseconds for transmition of 1 packet of size x).

nttcp
nuttcp
netperf
iperf

If I were to write a custom test, it would likely do the following:

Client:

1. Create a fake message of the specified length with some padding for
   an incrementing identifier. Open a TCP connection (with the right
   parameters like TCP_NODELAY, etc)
2. Loop and send messages containing an incrementing identifier. Store the current system time (in nanos) associated with the identifier.
3. Listen for responses, record the current time and record the latency of that identifier asynchronously

Server

1. Listen for a connection
2. Echo back any message received

Assuming both boxes had the same setup (configs, cards, OS, CPU, etc), are relatively close to eachother in the network, and one knows roughly the network's latency contribution, one could get a rough estimate of the OS + NIC contribution and begin testing various configurations.

One could also get a more accurate picture of the latency by using a network sniffer and snooping on the lines between the two hosts, calculating the latency between the two sides for an ID, and then subtracting that from the internally measured latency.

So, does anyone know a tool or even some boilerplate code for testing this sort of thing?