Shell – n opposite of ‘tee’, to stream multiple pipes together

io-redirectionshell

I have several processes that are producing output on STDOUT and STDERR, which I have redirected to numbered file descriptors, and I want to collate all the output together into a single file.

I have naively tried

[input processes] | cat <3 <4 <5 2>&1 >[output file]

but of course, this does not work as cat will wait until it's STDIN pipe is closed before reading data from any of the subsequent ones, causing my process to hang when the other pipes' buffers become full.

Any suggestions?

Best Answer

Collating output together is not really the dual of tee. tee makes multiple copies of its input, whereas collating output does not involve any merging of data.

To merge output sources, just redirect them all to the same file descriptor. The interleaving of the sources is somewhat unpredictable in general, but sufficiently small writes to a pipe are guaranteed to be atomic. (Being able to tell the boundaries from the read side is another story.)

{ data_source_1 &
  data_source_2 &
  wait; } >merged_output

If you're getting input from multiple file descriptors and you want to merge them, pass each of them through.

{ cat <&3 & cat <&4 & wait; } >merged_ouput

But usually you'd be able to redirect all the file descriptors to the same destination.

… 3>merged_ouput 4>&3