Text Processing – Does GNU grep’s -o Option Ignore Zero-Length Matches?

greppcretext processing

I found an answer on another site that was suggesting grep -oP '^\w+|$. I pointed out that the |$ is pointless in PCRE, since it just means "OR end of line" and will therefore always be true for regular lines. However, I can't exactly figure out what it does in GNU grep PCREs when combined with -o. Consider the following:

$ printf 'ab\na\nc\n\n' | perl -ne 'print if /ab|$/'
ab
a
c

$

(I am including the second prompt ($) character to show that the empty line is included in the results).

As expected, in Perl, that will match every line. Either because it contains an ab or because the $ matches the end of the line. GNU grep behaves the same way without the -o flag:

$ printf 'ab\na\nc\n\n' | grep -P 'ab|$'
ab
a
c

$

However, -o changes the behavior:

$ printf 'ab\na\nc\n\n' | grep -oP 'ab|$'
ab
$

This is the same as simply grepping for ab. The second part, the "OR end of line" seems to be ignored. It does work as expected without the -o flag:

What's going on? Does –o ignore 0-length matches? Is that a bug or is it expected?

Best Answer

My GNU grep man page says the following:

-o, --only-matching

Print only the matched (non-empty) parts of a matching line, with each such part on a separate output line.

emphasis is mine

I'm guessing it considers the end of line match to be an "empty match"

Related Solutions

Grep –color Adds ANSI Code ESC[K – How to Manage Displayed Text

You can change this behavior by setting the GREP_COLORS environment variable:

export GREP_COLORS=ne
echo -e "ab\rc" | grep --color=always "c"

From the grep man page:

          ne     Boolean  value  that prevents clearing to the end of line
                 using Erase in Line (EL) to Right  (\33[K)  each  time  a
                 colorized  item  ends.   This  is  needed on terminals on
                 which EL is not supported.  It  is  otherwise  useful  on
                 terminals  for  which  the back_color_erase (bce) boolean
                 terminfo capability  does  not  apply,  when  the  chosen
                 highlight colors do not affect the background, or when EL
                 is too slow or causes too much flicker.  The  default  is
                 false (i.e., the capability is omitted).

It's done in the first place to set the background of the rest of the line to the correct color, in case it was changed earlier (though by default it isn't; somebody might set it up to do so in their own settings).

You may also want to play with the other options that can be set in GREP_COLORS; see the man page for full details.

Grep – Using Perl-Compatible Regex Modifiers

PCRE's are not literally perl; they use a stand-alone C library, libpcre. Sometimes "PCRE style" is also used to refer to, e.g., other languages which implement regexps using the same/similar patterns, even though they do not rely on libpcre (note grep does use the actual libpcre).

Strictly speaking, a regular expression doesn't really include modifiers,¹ it's just a pattern. Some regexp interfaces (such as perl's) allow for the use of modifiers along with the patterns; grep doesn't.

There are a few differences in terms of the pattern semantics too. Again, keep in mind it's not actually perl, it's something based on it.

^{1. Unless they're actually inside the pattern, which you can do with grep. E.g. grep -P "(?i)foobar" will be a case insensitive match.}

Related Question