Progress Reports and Logging – Should They Go to stderr or stdout?

gnuposixstandardstderrstdout

Is there an official POSIX, GNU, or other guideline on where progress reports and logging information (things like "Doing foo; foo done") should be printed? Personally, I tend to write them to stderr so I can redirect stdout and get only the program's actual output. I was recently told that this is not good practice since progress reports aren't actually errors and only error messages should be printed to stderr.

Both positions make sense, and of course you can choose one or the other depending on the details of what you are doing, but I would like to know if there's a commonly accepted standard for this. I haven't been able to find any specific rules in POSIX, the GNU coding standards, or any other such widely accepted lists of best practices.

We have a few similar questions, but they don't address this exact issue:

So, are there any official rules on where progress reports and other informative messages (which aren't part of the program's actual output) should be printed?

Best Answer

Posix defines the standard streams thus:

At program start-up, three streams shall be predefined and need not be opened explicitly: standard input (for reading conventional input), standard output (for writing conventional output), and standard error (for writing diagnostic output). When opened, the standard error stream is not fully buffered; the standard input and standard output streams are fully buffered if and only if the stream can be determined not to refer to an interactive device.

The GNU C Library describes the standard streams similarly:

Variable: FILE * stdout
The standard output stream, which is used for normal output from the program.

Variable: FILE * stderr
The standard error stream, which is used for error messages and diagnostics issued by the program.

Thus, standard definitions have little guidance for stream usage beyond “conventional/normal output” and “diagnostic/error output.” In practice, it’s common to redirect either or both of these streams to files and pipelines, where progress indicators will be a problem. Some systems even monitor stderr for output and consider it a sign of problems. Purely auxiliary progress information is therefore problematic on either stream.

Instead of sending progress indicators unconditionally to either standard stream, it’s important to recognize that progress output is only appropriate for interactive streams. With that in mind, I recommend writing progress counters only after checking whether the stream is interactive (e.g., with isatty()) or when explicitly enabled by a command-line option. That’s especially important for progress meters that rely on terminal update behavior to make sense, like %-complete bars.

For certain very simple progress messages (“Starting X” ... “Done with X”) it’s more reasonable to include the output even for non-interactive streams. In that case, consider how users might interact with the streams, like searching with grep or paging with less or monitoring with tail -f. If it makes sense to see the progress messages in those contexts, they will be much easier to consume from stdout.

Related Question