Linux – Filter tail command through multiple grep commands to separate files

bashgreplinuxtail

What I am trying to do is filter the output of one log file into separated log files based on a grep filter.

tail -f test.log | tee >(grep "Error" > error.log) >(grep "Warning"" >
warning.log)

This way all "Error" entries are in one file, and all "Warning" entries are in a separate file.

I know this works in concept, because if I use cat instead of tail, I get the correct file output, but cannot track changes in real time (which I need because I am watching output logs from actively running tests)

Also, if I remove the '>' file re-directors from the grep commands it outputs the individual grep outputs to the console correctly; however, I want a recorded file as well.

Answer:

When writing to a file the output of grep was being buffered. Using egrep with a –line-buffer option fixed the behavior.

The new command looks like:

tail -f test.log | tee >(egrep –line-buffered "ERROR" > error.log) >(egrep –line-buffered "WARNING" > warning.log)

Best Answer

It might be easier if you use more than one line to do this. You could write a bash script:

#!/bin/bash

tail -f test.log | while read line; do
    if echo "$line" | grep -q "Error"; then
        echo "$line" >> error.log
    elif echo "$line" | grep -q "Warning"; then
        echo "$line" >> warning.log
    # The following is in case you want to print out lines that do not match 
    else
        echo "$line"
    fi
done
Related Question