Bash IO Redirection – Redirect stdout and stderr to File and Display stderr to Console

bashfilesio-redirection

I know how to redirect to a file, and use tee; on a basic level.
So

$ alias outanderr='bash -c "echo stdout >&1; echo stderr >&2"'
# A fake "application" displaying both output and error messages.

$ outanderr 1>file      # redirect stdout to a file, display stderr
stderr

$ outanderr 2>file      # redirect stderr to a file, display stdout
stdout

$ outanderr 1>file 2>&1 # redirect both to a file, display nothing

$ outanderr | tee file; echo "-- file contents --" && cat file
# redirect stdout to a file, display both (note: order is messed up)
stderr
stdout
-- file contents --
stdout

$ outanderr 2>&1 | tee file; echo "-- file contents --" && cat file
# redirect both to a file, display both
stdout
stderr
-- file contents --
stdout
stderr

The question is: what to write in place of the questionmarks to get the output below:

$ outanderr ???; echo "-- file contents --" && cat file
# redirect both to a file, display stderr
stderr
-- file contents --
stdout
stderr

Constaints:

  • Assuming bash.
  • The order should be kept in the file.
  • stderr contents are displayed in real time line by line, i.e. no buffering.
  • Separate script files can be used.
  • Magic may be necessary.

Best Answer

2>&1 >>outputfile | tee --append outputfile

For easy testing:

echo -n >outputfile; bash -c "echo stdout >&1; echo stderr >&2" 2>&1 >>outputfile |
  tee --append outputfile; echo "outputfile:"; cat outputfile

Edit 1:

This works by writing stdout (only) to the file, making sterr stdout so that it goes through the pipe, and having tee write its output to the same file.

Both writes must be done in append mode (>> instead of >) otherwise both would overwrite each others output.

As the pipe is a buffer there is no guarantee that the output appears in the file in the right order. This would not even change if an application was connected to both file descriptors (two pipes). For guaranteed order both outputs would have to go through the same channel and be marked respectively. Or you would need some really fancy stuff:

  1. If both stdout and stderr were redirected to a file (not the same file!) and both files were on a FUSE volume then the FUSE module could mark each single write with a timestamp so that a second application could sort the data correctly and combine it for the real output file. Or you don't mark the data but have the module create the combined output file. Most probably there is no FUSE module yet which does this...
  2. Both stdout and stderr could be directed to /dev/null. The outputs of the application would be separated by running it through strace -f -s 32000 -e trace=write. You would have to reverse the escaping in that case. Needless to say that the application does not run faster by being traced.
  3. Maybe the same could be reached by using an existing, simple FUSE module and tracing the module instead of the application. This may be faster than tracing the application because (or rather: if) the module probably has a lot less syscalls than the application.
  4. If the application itself can be modified: The app could be stopped after each output (but I think this is possible from the inside only) and continue only after receiving s signal (SIGUSR1 or SIGCONT). The application reading from the pipe would have to check both the pipe and the file for new data and to send the signal after each new data. Depending on the kind of application this may be faster or even slower than the strace method. FUSE would be the maximum speed solution.
Related Question