I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight the important results as they are displayed so the audience can get an idea of their frequency. The issue is that I can't get sed to operate on a running stream. It works fine if I put the results in a file, as in:
cat output.txt | sed "s/some text/some text bolded/"
But if I try the same thing on the running output, like this:
command | sed "s/some text/some text bolded/"
sed does nothing. Any thoughts?
As Lambert was helpful enough to point out, my saying that sed does nothing was vague. What is happening is that the program outputs to stdout (I'm pretty sure it's not writing to stderr) as it normally would, even if it's piped through sed.
The issue seems to be that the command calls a second program, which then outputs to stdout. There are a few lines printed by the first program; these I can edit. Then there is a stream of values printed by the second program; these I cannot edit.
Perl and awk methods do not work either.
Best Answer
Chances are that the command's output is buffered. When the command writes to a terminal, the buffer is flushed on every newline, so you see it appear at the expected rate. When the command writes to a pipe, the buffer is only flushed when it reaches a few kilobytes, so it lags a lot. Thus is the default behavior of the standard input/output library.
To force the command not to buffet its output, you can use
unbuffer(from expect) orstdbuf(from GNU coreutils).