I don't have much experience of using tee, so I hope this is not very basic.
After viewing one of the answers to this question I came across a strange beheviour with tee
.
In order for me to output the first line, and a found line, I can use this:
ps aux | tee >(head -n1) | grep syslog
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
syslog 806 0.0 0.0 34600 824 ? Sl Sep07 0:00 rsyslogd -c4
However, the first time I ran this (in zsh) the result was in the wrong order, the column headers were below the grep results (this did not happen again however), so I tried to swap the commands around:
ps aux | tee >(grep syslog) | head -n1
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
Only the first line is printed, and nothing else! Can I use tee to redirect to grep, or am I doing this in the wrong manner?
As I was typing this question, the second command actually worked once for me, I ran it again five times and then back to the one line result. Is this just my system? (I am running zsh within tmux).
Finally, why with the first command is "grep syslog" not shown as a result (there is only one result)?
For control here is the grep without the tee
ps aux | grep syslog
syslog 806 0.0 0.0 34600 824 ? Sl Sep07 0:00 rsyslogd -c4
henry 2290 0.0 0.1 95220 3092 ? Ssl Sep07 3:12 /usr/bin/pulseaudio --start --log-target=syslog
henry 15924 0.0 0.0 3128 824 pts/4 S+ 13:44 0:00 grep syslog
Update:
It seems that head is causing the whole command to truncate (as indicated in the answer below) the below command is now returning the following:
ps aux | tee >(grep syslog) | head -n1
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
syslog 806
Best Answer
The
grep
andhead
commands start at about the same time, and both receive the same input data at their own leisure, but generally, as data becomes available. There are some things that can introduce the 'unsynchronized' output which flips lines; for example:The multiplexed data from
tee
actually gets sent to one process before the other, depending primarily on the implementation oftee
. A simpletee
implementation willread
some amount of input, and thenwrite
it twice: Once to stdout and once to its argument. This means that one of those destinations will get the data first.However, pipes are all buffered. It is likely that these buffers are 1 line each, but they might be larger, which can cause one of the receiving commands to see everything it needs for output (ie. the
grep
ped line) before the other command (head
) has received any data at all.Notwithstanding the above, it's also possible that one of these commands receives the data but is unable to do anything with it in time, and then the other command receives more data and processes it quickly.
For example, even if
head
andgrep
are sent the data one line at a time, ifhead
doesn't know how to deal with it (or gets delayed by kernel scheduling),grep
can show its results beforehead
even gets a chance to. To demonstrate, try adding a delay:ps aux | tee >(sleep 1; head -n1) | grep syslog
This will almost certainly output thegrep
output first.I believe you often only get one line here, because
head
receives the first line of input and then closes its stdin and exits. Whentee
sees that its stdout has been closed, it then closes its own stdin (output fromps
) and exits. This could be implementation-dependent.Effectively, the only data that
ps
gets to send is the first line (definitely, becausehead
is controlling this), and maybe some other lines beforehead
&tee
close their stdin descriptors.The inconsistency with whether the second line appears is introduced by timing:
head
closes stdin, butps
is still sending data. These two events are not well-synchronized, so the line containingsyslog
still has a chance of making it totee
's argument (thegrep
command). This is similar to the explanations above.You can avoid this problem altogether by using commands that wait for all input before closing stdin/exiting. For example, use
awk
instead ofhead
, which will read and process all its lines (even if they cause no output):But note that the lines can still appear out-of-order, as above, which can be demonstrated by:
Hope this wasn't too much detail, but there are a lot of simultaneous things interacting with each other. Separate processes run simultaneously without any synchronization, so their actions on any particular run can vary; sometimes it helps to dig deep into the underlying processes to explain why.