Shell – Prepending each line with how long it took to generate it

shelltimestamps

I would like to have a script that would prepend each line of a stdin with information how long it took to generate it.
Basically for input:

foo
bar
baz

I would like to have

0 foo
10 bar
5 baz

Where 10 is 10 seconds passed between printing foo and printing bar, similar for 5, it took 5 seconds after printing bar to print baz.

I know there is a utility ts that shows timestamps and I know about https://github.com/paypal/gnomon, but I would prefer not to use javascript to do that.

Is there a standard tool for that or should I use awk and do processing?

Best Answer

Let's suppose that the script generating the output is called generate. Then, to display the number of seconds it takes for it to generate each line:

$ generate | ( t0=$(date +%s); while read -r line; do t1=$(date +%s); echo " $((t1-t0)) $line"; t0=$t1; done )
 2 foo
 0 foo
 5 foo
 3 foo

The same commands spread out over multiple lines looks like:

generate | ( t0=$(date +%s)
    while read -r line
    do
        t1=$(date +%s)
        echo " $((t1-t0)) $line"
        t0=$t1
    done
    )

Alternatively, for convenience, we can define a shell function that contains this code:

timer() { t0=$(date +%s); while read -r line; do t1=$(date +%s); echo " $((t1-t0)) $line"; t0=$t1; done; }

We can use this function as follows:

$ generate | timer
 0 foo
 2 foo
 4 foo
 3 foo

How it works

  • t0=$(date +%s)

    This captures the current time at the start of the script in seconds-since-epoch.

  • while read -r line; do

    This starts a loop which reads from standard input

  • t1=$(date +%s)

    This captures the time in seconds-since-epoch at which the current line was captured.

  • echo " $((t1-t0)) $line"

    This prints out the time in seconds that it took for the current line.

  • t0=$t1

    This updates t0 for the next line.

  • done

    This signals the end of the while loop.

Related Question