Bash – Trying to make sense of bash redirection syntaxes and their outputs

bashio-redirectionstderrstdout

I'm new to Linux and trying to understand how redirections work.

I have been testing various syntaxes for redirecting stdout and stderr to the same file, which don't all produce the same results.

For example, if I try to list 2 files that don't exist (file1 and file2) and 2 that do (foo and fz):

Syntax #1 (without redirection):

$ ls file1 foo fz file2

Here's the output I get in terminal:

ls: cannot access file1: No such file or directory
ls: cannot access file2: No such file or directory
foo  fz

Syntax #2:

Now, with redirection:

$ ls file1 foo fz file2 > redirect 2>&1

The redirect file contains the same as the result for Syntax #1:

ls: cannot access file1: No such file or directory
ls: cannot access file2: No such file or directory
foo
fz

So with both of the above syntaxes, it seems that the shell prints stderr first, then stdout.

Syntax #3:

Now, if I try with either of the following syntaxes:

$ ls file1 foo fz file2 > redirect 2> redirect

or

$ ls file1 foo fz file2 2> redirect > redirect

Then the redirect file will contain this:

foo
fz
nnot access file1: No such file or directory
ls: cannot access file2: No such file or directory

Here it looks like stdout is printed before stderr, but then we see that the beginning of stderr is "cropped" by the same number of characters as stdout.

The stdout is 6 chars long (foo fz, carriage return included), so the first 6 chars of the stderr (ls: ca) have been overwritten by stdout.
So it actually seems like stderr was printed first, and that stdout was then printed over stderr instead of being appended to it.

However, it would have made more sense to me if stderr had been completely erased and replaced with stdout, rather than just partially overwitten.

Syntax #4:

The only way I have found to correct Syntax #3 is by adding the append operator to the stdout:

$ ls file1 foo fz file2 >> redirect 2> redirect

or

$ ls file1 foo fz file2 2> redirect >> redirect

Which produces the same as Syntax #2:

ls: cannot access file1: No such file or directory
ls: cannot access file2: No such file or directory
foo
fz

This article here explains that Syntax #3 is wrong (presumably, so is Syntax #4). But for argument's sake: why is Syntax #3 wrong? What exactly is it telling (or not telling) the shell to do as opposed to Syntax #2?

Also, is there a reason why the output always displays stderr before stdout?

Thank you!

Best Answer

It's just like running two processes to write to the same file at the same time...bad idea. You wind up with two different open file handles and your data can get garbled (as it does in #3 above). Using syntax #2 is correct; it makes one file handle and points both stderr and stdout to the same place.

As for stderr always being printed first, there is no rule on this whatsoever. I suspect with ls it is because ls needs to check every entry in the directory before it can actually state that a particular file doesn't exist. So rather than make N passes over the directory table, it makes a single pass, checking for all command line arguments given, reports the errors, and prints the files it found. Other commands may print to stderr after stdout, or even alternate between them.

Related Question