Grep Piped Command Not Working – Troubleshooting Buffer Issues

buffergrep

I want to monitor a file. So I have tailed a file using below command, to execute a script per line.

tail -3f logfile.log | grep "action.*add" | sed -u -e "s/^/'/" -e "s/$/'/" | xargs -L 1 -P 5 bash testscript.sh

But it seems that script is not getting executed.
I observed that grep is not giving any input for next pipe.

When I tried,

tail -3f logfile.log | grep "action.*add"

it worked. But when given next filter like sed, grep, xargs etc. It didn't worke like the one shown below.

tail -3f /var/tmp/rabbitmq-tracing/logfile.log | grep "action.*add" | grep add

Please help me to know why this is happening and how to overcome this.

Edit 1:
Basically anything like below should work and it was working previously. Confused why it is not working now.

tail -f file.txt | grep something | grep something | grep something

EDIT 2:
Output line after first grep will be a json string like below. And I want to give this line as input( enclosed in single quotes) to bash script.

{"twNotif": {"originator": "api", "chain": "test", "txId": "08640-0050568a5514", "version": "1.0", "msgType": "api", "twData": {"api": {"hostId": "007bdcc5", "user": "test", "cmdTxt": "100599"}}, "action": "add", "store": "test", "msgTime": 1467280648.971042}}

Best Answer

use --line-buffered switch on grep

tail -3f logfile.log | grep --line-buffered "action.*add" | sed -u -e "s/^/'/" -e "s/$/'/" | xargs -L 1 -P 5 bash testscript.sh

from man grep:

--line-buffered Use line buffering on output. This can cause a performance penalty.

or you can use stdbuf read more

stdbuf allows one to modify the buffering operations of the three standard I/O streams associated with a program. Synopsis:

use this syntax:

... | stdbuf -oL grep ... | ...

your example:

tail -3f logfile.log | stdbuf -oL grep "action.*add" | sed -u -e "s/^/'/" -e "s/$/'/" | xargs -L 1 -P 5 bash testscript.sh
Related Question