Grep – How to Filter Output of Command by Color

colorsfiltergreptext;

I am running a utility that doesn't offer a way to filter its output. Nothing in the text of the output indicates that a particular function failed but it does show in red. The output is so long that at the end when it reports some # of errors I can't always scroll to see the output where the error occurred.

How can I filter out non-red text?

pseudo code:

dolongtask | grep -color red

Edit

The command outputs other colors as well and I need to be able to filter out all text that isn't red. Also the text coloring is multiline.

Best Answer

Switching the color is done through escape sequences embedded in the text. Invariably, programs issue ANSI escape sequences, because that's what virtually all terminals support nowadays.

The escape sequence to switch the foreground color to red is \e[31m, where \e designates an escape character (octal 033, hexadecimal 1b, also known as ESC, ^[ and various other designations). Numbers in the range 30–39 set the foreground color; other numbers set different attributes. \e[0m resets all attributes to their default value. Run cat -v to check what the program prints, it might use some variant such as \e[0;31m to first reset all attributes, or \e[3;31 to also switch italics on (which many terminals don't support).

In ksh, bash or zsh, you can use $'…' to enable backslash escapes inside the quotes, which lets you type $'\e' to get an escape character. Note that you will then have to double any backslash that you want to pass to grep. In /bin/sh, you can use "$(printf \\e)" or type a literal escape character.

With the GNU grep -o option, the following snippet filters red text, assuming that it starts with the escape sequence \e[31m, ends with either \e[0m or \e[30m on the same line, and contain no embedded escape sequence.

grep -Eo $'\e\\[31m[^\e]*\e\\[[03]?m'

The following awk snippet extracts red text, even when it's multiline.

awk -v RS='\033' '
    match($0, /^\[[0-9;]*m/) {
        color = ";" substr($0, 2, RLENGTH-2) ";";
        $0 = substr($0, RLENGTH+1);
        gsub(/(^|;)0*[^03;][0-9]*($|;)/, ";", color);
        red = (color ~ /1;*$/)
    }
    red'

Here's a variation which retains the color-changing commands, which could be useful if you're filtering multiple colors (here red and magenta).

awk -v RS='\033' '
    match($0, /^\[[0-9;]*m/) {
        color = ";" substr($0, 2, RLENGTH-2) ";";
        printf "\033%s", substr($0, 1, RLENGTH);
        $0 = substr($0, RLENGTH+1);
        gsub(/(^|;)0*[^03;][0-9]*($|;)/, ";", color);
        desired = (color ~ /[15];*$/)
    }
    desired'

Related Solutions

Ubuntu – Is GREP_OPTIONS= –color=always ignored

If grep o produces color output, then either grep is an alias to grep --color=auto or grep --color=always (or possibly more options), or GREP_OPTIONS is set to a value that contains --color=auto or --color=always. Since $GREP_OPTIONS is empty, it must be the alias.

Since grep o | less -R doesn't show colors, the alias must be to grep --color=auto (a sensible choice). With the alias, the grep command always receives the --color option on the command line, and this takes precedence over the environment variable.

If you want to use the environment variable, remove the alias definition from your ~/.bashrc, or for one session run unalias grep. You can replace alias grep='grep --color=auto' by export GREP_OPTIONS='--color=auto': they have essentially the same meaning, except that:

setting GREP_OPTIONS to a different value only overrides the latter;
the alias only kicks in when you run grep from an interactive shell, whereas setting GREP_OPTIONS also applies when grep is run from scripts and other applications.

Never put --color=always or most other options in GREP_OPTIONS: it would break many programs that parse the output of grep. --color=auto is about the only safe option to put in GREP_OPTIONS. For anything else, use the alias. Future versions of GNU grep will drop support for the option for this reason.

Note that the alias definition goes into ~/.bashrc (it's a shell setting), whereas the environment variable definition goes into ~/.profile (it's a session setting). See Is there a ".bashrc" equivalent file read by all shells?

If you want to run the unaliased command just once, run \grep instead of grep (quoting any part of the name bypasses the alias lookup).

How to grep MATCH colored input

Let's use tput to generate the color code for your terminal for yellow and black:

$ yel=$(tput setaf 3)
$ blk=$(tput setaf 0)

Let's examine what the yellow code actually includes:

$ echo -n "$yel" | hexdump -C
00000000  1b 5b 33 33 6d                                    |.[33m|
00000005

Now, we can use grep to search for the yellow color code and print the string that matches from the beginning of the yellow code to the next code, whatever that code is:

$ echo "abc ${yel}Yellow${blk} def" | grep -Eo $'\x1b\[33m.[^\x1b]*\x1b\[....'
Yellow

Note that the color code for yellow includes [ which grep considers to be a regex active character. Thus, to match a literal [, we need to escape it for grep. To do this, we use bash's $'...' to define the color code with [` escaped.

It is likely that there is more than one code to produce yellow on your terminal. You will want to examine the output of whatever command you are using to determine what codes are being used and include them in your grep command.

Edit

Best Answer

Related Solutions

Ubuntu – Is GREP_OPTIONS= –color=always ignored

How to grep MATCH colored input

Related Question