Shell – How to Filter Out Lines of Command Output That Occur in a Text File

filterpipeshell

Let's say we have a text file of forbidden lines forbidden.txt. What is a short way to filter all lines of a command output that exist in the text file?

cat input.txt | exclude-forbidden-lines forbidden.txt | sort

Best Answer

Use grep like this:

$ grep -v -x -F -f forbidden.txt input.txt

That long list of options to grep means

  • -v Invert the sense of the match, i.e. look for lines not matching.
  • -x When matching a pattern, require that the pattern matches the whole line, i.e. not just anywhere on the line.
  • -F When matching a pattern, treat it as a fixed string, i.e. not as a regular expression.
  • -f Read patterns from the given file (forbidden.txt).

Then pipe that to sort or whatever you want to do with it.

Related Question