Capture all output, error, warning and their appearance order of linux’s command

command lineio-redirection

In linux we can redirect stdout and stderr or both of them to files. For example:

Redirect stdout and stderr to two files: command 1>out 2>err. By this way, we can separate output and error in two difference files.
Redirect both stdout and stderr to files: command &>all. By this way, we can capture the appearance orders of output and error message.

I want to combine the two above command: command 1>out 2>err &>all, to capture the appearance order of output and error, and separate output and error. But the above command does not work, the out and err files are empty. If I reverse the order, the all file will be empty.
So, is there anyway to combine the two command above as I want?

Best Answer

In bash it's a bit tricky:

command 2> >(tee err) 1> >(tee out) | tee >all

Here we need process substitution >(...) and tee to work around the problem. With process substitution the tee process is attachted to the corresponding channel. tee then writes the lines into the file and prints then to the STDOUT. So after writing to the files, both STDERR and STDOUT will be printed to STDOUT. In bash, we cannot use multiple redirections of the same chanel in one command. That's why we need to pipe that output to tee again, which then just prints it.

Within zsh (notice the option MULTIOS must be set, which is detault in zsh):

command >out 2>err &>all

Related Solutions

Bash – How to Redirect Bash stdout+stderr to One File and stderr to Another File

With zsh (and zsh only) and its multios feature:

your-cmd 2> stdout+stderr.log >&2 2> stderr.log

As fd 2 is redirected twice, zsh implements an internal tee to have it sent to both files.

With bash (or any Bourne-like shell (other than zsh where you'd need to disable multios for it to work here)), you can do the teeing by hand with:

{ your-cmd 2>&1 >&3 3>&- | tee stderr.log 3>&-; } > stderr+stdout.log 3>&1

(though you lose the exit status of your-cmd. zsh has it in $pipestatus[1], bash in "${PIPESTATUS[0]}" though (provided the redirection into stderr+stdout.log didn't fail)).

To record the pid of your-cmd, you could do:

{ sh -ec 'echo "$$" > /var/run/pidfile; exec your-cmd' 2>&1 >&3 3>&- |
   tee stderr.log 3>&-; } > stderr+stdout.log 3>&1

With yash and it's process redirection feature:

your-cmd > stdout+stderr.log 2>(tee stderr.log)

(but note that yash will not wait for the termination of that tee command, so the log files may not be complete by the time you run the next command after that).

Something similar (and with the same caveat) can be done with process substitution in bash, zsh and ksh93:

{ your-cmd 2> >(tee stderr.log); } > stderr+stdout.log

To run in background and get the pid:

(exec your-cmd 2> >(tee stderr.log)) > stderr+stdout.log & pid=$!

With rc:

{your-cmd |[2=0] tee stderr.log} > stdout+stderr.log

rc's pipes allow specifying which file descriptors are connected to the pipe. With other shells, it's always fd 1 of the left command and fd 0 of the right one (hence the little dance with fd 3 above to shift file descriptors around). rc will report failure if either your-cmd or tee fails, though the exact number may be lost.

Bash – Ambiguous output redirect

If you're running (t)csh, you get Ambiguous output redirect. if you try to set up two conflicting redirections:

> echo foo > a > b
Ambiguous output redirect.

In Bash, you could get a similar error if use an array with multiple elements in place of the filename:

$ set aa bb
$ echo foo > "$@"
bash: "$@": ambiguous redirect

As mentioned in answers to stderr redirection not working in csh, the >& operator works in (t)csh to redirect both stdout and stderr. 2>&1 is the standard way to redirect stderr to the same place as stdout, but (t)csh doesn't support that. Instead, it takes the combination > foo 2>&1 as a redirection to foo, a regular argument 2, and a redirection to 1, and the redirections conflict, so you get the error.

>& also works in Bash and zsh, but isn't a standard feature.

Best Answer

Related Solutions

Bash – How to Redirect Bash stdout+stderr to One File and stderr to Another File

Bash – Ambiguous output redirect

Related Question