Why doesn’t grep ignore binary files by default

grep

The manpage for grep describes the -I flag as follows:

-I      Ignore binary files.  This option is equivalent to 
        --binary-file=without-match option.

It also says this about binary files:

 --binary-files=value Controls searching and printing of binary files.
         Options are binary, the default: search binary files but do not print
         them; without-match: do not search binary files; and text: treat all
         files as text.

I cannot think of a scenario where I would care about matches in binary files. If such a scenario exists, surely it must be the exception rather than the norm. Why doesn't grep ignore binary files by default rather than requiring setting this flag to do so?

Best Answer

Not everything that grep thinks is a binary file, is actually a binary file. e.g. puppet's logs have ansi color coding in them, which makes grep think they're binary. I'd still want to search them if I'm grepping through /var/log though.

Related Solutions

Grep and Escaping a Dollar Sign

There's 2 separate issues here.

grep uses Basic Regular Expressions (BRE), and $ is a special character in BRE's only at the end of an expression. The consequence of this is that the 2 instances of $ in $Id$ are not equal. The first one is a normal character and the second is an anchor that matches the end of the line. To make the second $ match a literal $ you'll have to backslash escape it, i.e. $Id\$ . Escaping the first $ also works: \$Id\$, and I prefer this since it looks more consistent.¹
There are two completely unrelated escaping/quoting mechanisms at work here: shell quoting and regex backslash quoting. The problem is many characters that regular expressions use are special to the shell as well, and on top of that the regex escape character, the backslash, is also a shell quoting character. This is why you often see messes involving double backslashes, but I do not recommend using backslashes for shell quoting regular expressions because it is not very readable.

Instead, the simplest way to do this is to first put your entire regex inside single quotes as in 'regex'. The single quote is the strongest form of quoting the shell has, so as long as your regex does not contain single quotes, you no longer have to worry about shell quoting and can focus on pure BRE syntax.

So, applying this back to your original example, let's throw the correct regex (\$Id\$) inside single quotes. The following should do what you want:

grep '\$Id\$' my_dir/my_file

The reason \$Id\$ does not work is because after shell quote removal (the more correct way of saying shell quoting) is applied, the regex that grep sees is $Id$ . As explained in (1.), this regex matches a literal $Id only at the end of a line because the first $ is literal while the second is a special anchor character.

^{¹ Note also that if you ever switch to Extended Regular Expressions (ERE), e.g. if you decided to use egrep (or grep -E), the $ character is always special. In ERE's $Id$ would never match anything because you can't have characters after the end of a line, so \$Id\$ would be the only way to go.}

Grep to ignore patterns

I'm not fully following your example + the description but it sounds like what you want is this:

$ grep -v "<a href=.*title=.*NOTNEEDED" sample.txt 
<a href="http://website1.com" title="something">
<a href="http://website1.com" information="something" title="something">
<a href="http://website2.com" title="some_other_thing">
<a href="http://website2.com" information="something" title="something">

So for your example:

$ curl www.example.com | grep -v "<a href=.*title=" | grep -v NOTNEEDED > new.txt

Best Answer

Related Solutions

Grep and Escaping a Dollar Sign

Grep to ignore patterns

Related Question