Linux Command Line Tools for Analyzing OS + NIC TCP Latency

command linelinuxnetworkingtcp

I'm hoping to find an existing Linux tool for measuring latency:

I'm trying to diagnose how much latency my OS and NIC card are adding to TCP latency in Linux. I looked at the following tools but all (at least at the surface level) measured bandwidth (Mbps/sec) rather than latency (nanoseconds for transmition of 1 packet of size x).


If I were to write a custom test, it would likely do the following:


  1. Create a fake message of the specified length with some padding for
    an incrementing identifier. Open a TCP connection (with the right
    parameters like TCP_NODELAY, etc)
  2. Loop and send messages containing an incrementing identifier. Store the current system time (in nanos) associated with the identifier.
  3. Listen for responses, record the current time and record the latency of that identifier asynchronously


  1. Listen for a connection
  2. Echo back any message received

Assuming both boxes had the same setup (configs, cards, OS, CPU, etc), are relatively close to eachother in the network, and one knows roughly the network's latency contribution, one could get a rough estimate of the OS + NIC contribution and begin testing various configurations.

One could also get a more accurate picture of the latency by using a network sniffer and snooping on the lines between the two hosts, calculating the latency between the two sides for an ID, and then subtracting that from the internally measured latency.

So, does anyone know a tool or even some boilerplate code for testing this sort of thing?

Best Answer

There is a tool called qperf that can provide you such statistics. Though, it is bound to OFED/RDMA packages.

# qperf -v xx.xx.xx.xx tcp_lat udp_lat
    latency        =  29.2 us
    msg_rate       =  34.2 K/sec
    loc_cpus_used  =    41 % cpus
    rem_cpus_used  =  11.5 % cpus
    latency        =  24.8 us
    msg_rate       =  40.4 K/sec
    loc_cpus_used  =    34 % cpus
    rem_cpus_used  =     4 % cpus
Related Question