How to read lines at a fixed speed

bandwidthiolimitnetcattcp

I need to read a large log file and send it over a local network using (netbsd) netcat between two VMs on the same host workstation.

I know that netcat has an interval, but as far as I can tell, the smallest interval you can use is 1 line/second.

Most of the files I need to send this way have hundreds of thousands of lines, and some close to a million lines, so one line per second isn't feasible.

If I just use cat, my host computer/workstation winds up getting bogged down to the point of being unusable.

Using bash and common *nix tools, is there a way I can send the files, but feed it to netcat at a rate of say, 5-10 lines/second or something like that?

The end goal of this is to allow me to do some proof of concept testing for a centralized log database I am considering.

Best Answer

If you use bash and pipes, and are looking for an easy and dirty solution, you can try using sleep.

You can use this which act like cat but with a pause at each line. while read i; do echo "$i"; sleep 0.01; done. Here is an example at a little less than 100 lines per second.

$ time (seq 1 100 | while read i; do echo "$i"; sleep 0.01; done)
[...]
real    0m1.224s
user    0m0.012s
sys     0m0.052s
Related Question