Grep – Is It Possible to Grep from a Stream That Has No New Line?

grepnewlines

When running this code:

while true; do printf "Match Unmatch\n"; sleep 1s; done | grep -o "Match"

in a terminal, grep finds the word Match. I get the output:

Match
Match
Match
...

However, when I run the following code: 

while true; do printf "Match Unmatch"; sleep 1s; done | grep -o "Match"

then grep seems to find nothing, at least it prints nothing. Is grep waiting for a newline that will never come? Is there some way of making grep, sed or similar tool capture matches in streams without the newline character?

Best Answer

Grep mainly looks for lines that contain a match for the given pattern. Depending on the pattern, it may or may not be possible to determine whether a line matches without looking at the whole line. With grep Match, this is possible, but with grep 'Match$', it isn't. With grep -o Match, grep could print Match as soon as it sees it, but with grep -E -o '(Match)+', if grep has read MatchMa, it doesn't know whether tch will follow or not.

Grep does not implement the special cases where it would be possible to write some output before seeing a whole line. (I think it doesn't implement any such special case, but I'm not completely sure: GNU grep has several modes depending on the pattern that behave somewhat differently.) It just reads one whole line before trying to match.

If there's a character that's never appears in matching text and that always (or at least often) appear between matches, transform this character into a newline. (Or into a null byte and use grep -z.)

while true; do printf "Match Unmatch"; sleep 1s; done | stdbuf -o0 tr ' ' '\n' | grep -o "Match"

Note the use of stdbuf -o0 to prevent tr from buffering its output. And if you pipe the output of grep, you'll need that for grep as well, or use grep --line-buffered. (--line-buffered buffers the output of grep; it has no impact on how grep reads input, and it's on by default when grep is printing to a terminal.)

Related Question