Tail -f a file for 10 minutes / until N matching lines

tail

I've got a log file that I process as follows:

grep pattern /var/log/whatever.log | \
    cut ... | sort | uniq -c | sort -rn | \
    etc....

However, the log file is quite big, and records events from the beginning of the day. It's also appended to constantly. I'd like to only process the ~~last~~ next 10 minutes of the file.

So I'm looking for something like the following:

killafter 600 tail -f /var/log/whatever.log | stuff

Or, better yet (wait however long it takes to capture 1000 matching lines):

tail -f /var/log/whatever.log | grep pattern | stopafter -lines 1000 | stuff

Are there any tools that'll let me do this?

Best Answer

roaima rightly points you at the timeout command, and head actually terminates after it has read the desired number of lines, so I would hope that with

timeout 600s tail -f ‹logfile› | ‹filter› | head -n 1000

you'd get there.

Related Solutions

Bash – How to limit the number of lines a command’s output has available in bash

If you do not want to limit the scrolling region (see my other answer), you can also use the carriage return to go back to the beginning of the line before printing the next line. There is an escape sequence that clears the rest of the line, which is necessary when the current line is shorter than the previous line.

nohup wget http://cdimage.debian.org/debian-cd/6.0.3/amd64/iso-dvd/debian-6.0.3-amd64-DVD-1.iso &
el="$(tput el)"; # Clear to the end of the line
tail -n 1 -f nohup.out | while read -r line; do echo -n $'\r'"$el$line"; done;

Tail Command – Using ‘tail -f’ Until Text is Seen

You can pipe the tail -f into sed, telling it to quit when it sees the line you're searching for:

tail -f /path/to/file.log | sed '/^Finished: SUCCESS$/ q'

sed will output each line it processes by default, and exit after it sees that line. The tail process will stop when it tries to write the next line and sees its output pipe is broken

Best Answer

Related Solutions

Bash – How to limit the number of lines a command’s output has available in bash

Tail Command – Using ‘tail -f’ Until Text is Seen

Related Question