How to color specific parts of the rsync output

colorsgreprsyncsed

I run this command to rsync some files from Windows (Cygwin) to my local NAS:

rsync -PaSq --delete -e "/cygdrive/C/cygwin64/bin/ssh -i keyfile -p XXXX" "/source/" admin@192.168.1.1:/destination/

I now want to highlight in red any error messages. By that I mean highlighting the errors from rsync once it connects and not the ssh connection errors.

So this ssh connection error message remains unchanged:

ssh_exchange_identification: Connection closed by remote host
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.2]

But this is highlighted in red:

rsync: delete_file: unlink(test/test.txt) failed: Permission denied (13)

I've tried the following with no luck.
Any ideas appreciated.

rsync -PaSq --delete -e "/cygdrive/C/cygwin64/bin/ssh -i keyfile -p XXXX" "/source/" admin@192.168.1.1:/destination/ | tput setaf 1; sed -n '/rsync:/p'

rsync -PaSq --delete -e "/cygdrive/C/cygwin64/bin/ssh -i keyfile -p XXXX" "/source/" admin@192.168.1.1:/destination/ | egrep --color '.*rsync:.*|$'

rsync -PaSq --delete -e "/cygdrive/C/cygwin64/bin/ssh -i keyfile -p XXXX" "/source/" admin@192.168.1.1:/destination/ | grep --color=auto '.*rsync:.*|$'

Best Answer

The problem you face comes from the fact that rsync outputs errors to the standard error (file descriptor 2), but the pipe by default transfers only the standard output (file descriptor 1), leaving standard error untouched. So the command after the pipe (sed or grep in your example) doesn't see errors which came from rsync.

You can redirect both stdout and stderr using |& aka 2>&1 |. Another approach is to redirect only stderr, leavng stdout where it pointed, so try something like:

rsync ... |& grep --color '.*rsync:.*'

{ rsync ...  2>&1 >&3 3>&- | grep --color '.*rsync:.*' 3>&-; } 3>&1

Second approach is a little bit complicated because we need to temporarily redirect stdout somewhere (to descriptor 3 in this case) in order to not pass it to the grep.

Related Solutions

How to examine the fast output of rsync

Redirect the error stream to a file that you can read later. The verbose output goes to standard output whereas errors go on standard error.

sudo rsync -azvv /home/path/folder1/ /home/path/folder2 2>rsync-errors
echo $?
cat rsync-errors

Alternatively, instead of using verbose mode, use --progress or -P to just see a progress indicator, that way you will see error messages scroll by. It is possible that they'll scroll too fast if there are a lot of them.

In any case, check that the return status of rsync is 0, indicating that there were no errors.

Shell – How to keep color codes with grep input

Use --color=always.

grep detects if output is to a pipe (or file). You most never want colors when output is to file – as that is escape sequences for the terminal. Typically:

foo ^[[01;31m^[[Kbar^[[m^[[K

from e.g.:

grep pattern file > result

To override use --color=always.

grep --color=always pattern file | ...

Example:

Having file:

ID=111;Year=2013;foo=bar
ID=222;Year=2013;foo=baz

Then

grep --color=always ID file | grep --color=always 2013 | grep foo

would color ID, 2013 and foo.

grep --color=always ID file | grep 2013 | grep foo

would color ID and foo, but not 2013.

¹Important: You also have to remember that the added clutter from colors is passed to next command in the chain. Once ID is colored, you can't (with ease), match e.g. ID=111.

On some occasion one would perhaps want terminal colors in file. Try e.g.

 grep --color=always foo file > result
 cat result

Though the resulting file would have very limited portability.

The coloring itself is also an extension.

As mentioned by the good @slm, you could add:

export GREP_OPTIONS="--color=always"

to your .bashrc etc, but don't unless you for some reason really understand the implications and still want to do it. It would in many ways break grep due to the fact mentioned above ¹.

Add an alias if you use it often.

 alias cgrep='grep --color=always'

GREP_OPTIONS and --color take three options: never, auto and always. The two first should be the only ones considered for GREP_OPTIONS.

You could also check out GREP_COLORS in the man pages or at gnu grep.

Best Answer

Related Solutions

How to examine the fast output of rsync

Shell – How to keep color codes with grep input

Related Question