Shell – How to Grep for Multiple Patterns with Pipe Character

grepquotingregular expressionshell

I want to find all lines in several files that match one of two patterns. I tried to find the patterns I'm looking for by typing

grep (foo|bar) *.txt

but the shell interprets the | as a pipe and complains when bar isn't an executable.

How can I grep for multiple patterns in the same set of files?

Best Answer

First, you need to protect the pattern from expansion by the shell. The easiest way to do that is to put single quotes around it. Single quotes prevent expansion of anything between them (including backslashes); the only thing you can't do then is have single quotes in the pattern.

grep -- 'foo*' *.txt

(also note the -- end-of-option-marker to stop some grep implementations including GNU grep from treating a file called -foo-.txt for instance (that would be expanded by the shell from *.txt) to be taken as an option (even though it follows a non-option argument here)).

If you do need a single quote, you can write it as '\'' (end string literal, literal quote, open string literal).

grep -- 'foo*'\''bar' *.txt

Second, grep supports at least¹ two syntaxes for patterns. The old, default syntax (basic regular expressions) doesn't support the alternation (|) operator, though some versions have it as an extension, but written with a backslash.

grep -- 'foo\|bar' *.txt

The portable way is to use the newer syntax, extended regular expressions. You need to pass the -E option to grep to select it (formerly that was done with the egrep separate command²)

grep -E -- 'foo|bar' *.txt

Another possibility when you're just looking for any of several patterns (as opposed to building a complex pattern using disjunction) is to pass multiple patterns to grep. You can do this by preceding each pattern with the -e option.

grep -e foo -e bar -- *.txt

Or put patterns on several lines:

grep -- 'foo
bar' *.txt

Or store those patterns in a file, one per line and run

grep -f that-file -- *.txt

Note that if *.txt expands to a single file, grep won't prefix matching lines with its name like it does when there are more than one file. To work around that, with some grep implementations like GNU grep, you can use the -H option, or with any implementation, you can pass /dev/null as an extra argument.

^{¹ some grep implementations support even more like perl-compatible ones with -P, or augmented ones with -X, -K for ksh wildcards...}

^{² while egrep has been deprecated by POSIX and is sometimes no longer found on some systems, on some other systems like Solaris when the POSIX or GNU utilities have not been installed, then egrep is your only option as its /bin/grep supports none of -e, -f, -E, \| or multi-line patterns}

Related Solutions

Using xargs to grep multiple patterns

Yes, find ./work -print0 | xargs -0 rm will execute something like rm ./work/a "work/b c" .... You can check with echo, find ./work -print0 | xargs -0 echo rm will print the command that will be executed (except white space will be escaped appropriately, though the echo won't show that).

To get xargs to put the names in the middle, you need to add -I[string], where [string] is what you want to be replaced with the argument, in this case you'd use -I{}, e.g. <strings.txt xargs -I{} grep {} directory/*.

What you actually want to use is grep -F -f strings.txt:

-F, --fixed-strings
  Interpret PATTERN as a  list  of  fixed  strings,  separated  by
  newlines,  any  of  which is to be matched.  (-F is specified by
  POSIX.)
-f FILE, --file=FILE
  Obtain  patterns  from  FILE,  one  per  line.   The  empty file
  contains zero patterns, and therefore matches nothing.   (-f  is
  specified by POSIX.)

So grep -Ff strings.txt subdirectory/* will find all occurrences of any string in strings.txt as a literal, if you drop the -F option you can use regular expressions in the file. You could actually use grep -F "$(<strings.txt)" directory/* too. If you want to practice find, you can use the last two examples in the summary. If you want to do a recursive search instead of just the first level, you have a few options, also in the summary.

Summary:

# grep for each string individually.
<strings.txt xargs -I{} grep {} directory/*

# grep once for everything
grep -Ff strings.txt subdirectory/*
grep -F "$(<strings.txt)" directory/*

# Same, using file
find subdirectory -maxdepth 1 -type f -exec grep -Ff strings.txt {} +
find subdirectory -maxdepth 1 -type f -print0 | xargs -0 grep -Ff strings.txt

# Recursively
grep -rFf strings.txt subdirectory
find subdirectory -type f -exec grep -Ff strings.txt {} +
find subdirectory -type f -print0 | xargs -0 grep -Ff strings.txt

You may want to use the -l option to get just the name of each matching file if you don't need to see the actual line:

-l, --files-with-matches
  Suppress  normal  output;  instead  print the name of each input
  file from which output would normally have  been  printed.   The
  scanning  will  stop  on  the  first match.  (-l is specified by
  POSIX.)

Grep – How to ‘Grep’ Patterns Across Multiple Lines

Here's a sed one that will give you grep-like behavior across multiple lines:

sed -n '/foo/{:start /bar/!{N;b start};/your_regex/p}' your_file

How it works

-n suppresses the default behavior of printing every line
/foo/{} instructs it to match foo and do what comes inside the squigglies to the matching lines. Replace foo with the starting part of the pattern.
:start is a branching label to help us keep looping until we find the end to our regex.
/bar/!{} will execute what's in the squigglies to the lines that don't match bar. Replace bar with the ending part of the pattern.
N appends the next line to the active buffer (sed calls this the pattern space)
b start will unconditionally branch to the start label we created earlier so as to keep appending the next line as long as the pattern space doesn't contain bar.
/your_regex/p prints the pattern space if it matches your_regex. You should replace your_regex by the whole expression you want to match across multiple lines.

Best Answer

Related Solutions

Using xargs to grep multiple patterns

Grep – How to ‘Grep’ Patterns Across Multiple Lines

Related Question