Linux – Name of the Process on the Other End of a Unix Pipe

linuxpipeprocprocess

If two processes are connected by a pipe,

> cmd1 | cmd2

is there any way for cmd1 to find out the name (or PID) of the process on the other side of the pipe (cmd2)?

Also, vice versa, is there any way for cmd2 to get the name/PID of cmd1?

I know that there is isatty(3) to check if the output goes to (or the input comes from) a terminal, so I wondered if there is a way to find out a little bit more about the other hand side.

Best Answer

You can see the pipe in /proc/$PID/fd. The descriptor is a symlink to something like pipe:[188528098]. With that information you can search for the other process:

$ lsof -n | grep -w 188528098
sleep      1565   hl    1w     FIFO    0,12     0t0  188528098 pipe
sleep      1566   hl    0r     FIFO    0,12     0t0  188528098 pipe

Or, if you want to be sure (for automatic processing) that the number is the socket and not part of a file name:

$ lsof -n | awk 'NF==9 && $5=="FIFO" && $9=="pipe" && $8==188528098'

With lsof 4.88 and above, you can also use the -E or +E flags:

In combination with -p <pid>, -d <descriptor>, you can get the endpoint information for a specific descriptor of a given pid.

$ sleep 1 | sh -c 'lsof -E -ap "$$" -d 0; exit'
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
sh      27176 chazelas    0r  FIFO   0,10      0t0 2609460 pipe 27175,sleep,1w

Above telling us that fd 0 of sh is a pipe with fd 1 of sleep at the other end. If you change -E to +E, you also get the full information for that fd of sleep:

$ sleep 1 | sh -c 'lsof +E -ap "$$" -d 0; exit'
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
sleep   27066 chazelas    1w  FIFO   0,10      0t0 2586272 pipe 27067,sh,0r 27068,lsof,0r
sh      27067 chazelas    0r  FIFO   0,10      0t0 2586272 pipe 27066,sleep,1w

(see how lsof also has the pipe on its stdin)

Related Solutions

Way to identify which process turns into Zombie process

The audit subsystem of the Linux kernel can be very useful to figure out what processes are becoming zombie processes. I just had the following situation:

server ~ # ps -ef --forest
[...]
root     16385     1  0 17:04 ?        00:00:00 /usr/sbin/apache2 -k start
root     16388 16385  0 17:04 ?        00:00:00  \_ /usr/bin/perl -T -CSDAL /usr/lib/iserv/apache_user
root     16389 16385  0 17:04 ?        00:00:00  \_ /usr/bin/perl -T -CSDAL /usr/lib/iserv/apache_user
www-data 16415 16385  0 17:04 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 18254 16415  0 17:23 ?        00:00:00  |   \_ [sh] <defunct>
www-data 18347 16415  0 17:23 ?        00:00:00  |   \_ [sh] <defunct>
www-data 22966 16415  0 18:18 ?        00:00:00  |   \_ [sh] <defunct>
www-data 16583 16385  0 17:05 ?        00:00:01  \_ /usr/sbin/apache2 -k start
www-data 18306 16583  0 17:23 ?        00:00:00  |   \_ [sh] <defunct>
www-data 18344 16583  0 17:23 ?        00:00:00  |   \_ [sh] <defunct>
www-data 17561 16385  0 17:12 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 22983 17561  0 18:18 ?        00:00:00  |   \_ [sh] <defunct>
www-data 18318 16385  0 17:23 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 19725 16385  0 17:43 ?        00:00:01  \_ /usr/sbin/apache2 -k start
www-data 22638 16385  0 18:13 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 22659 16385  0 18:14 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 25102 16385  0 18:41 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 25175 16385  0 18:42 ?        00:00:00  \_ /usr/sbin/apache2 -k start
www-data 25272 16385  0 18:44 ?        00:00:00  \_ /usr/sbin/apache2 -k start

The cause for these zombie processes is most probably a PHP script, but as these Apache child processes are processing lots of HTTP requests and lots of different PHP scripts, it's very hard to figure out which one could be responsible. Linux has also already deallocated important information of these zombie processes, so we don't even have /proc/<pid>/cmdline to figure out which script or -c command /bin/sh may have been running:

server ~ # cat /proc/18254/cmdline 
server ~ #

To figure it out, I've installed auditd: https://linux-audit.com/configuring-and-auditing-linux-systems-with-audit-daemon/

I set up the following audit rules:

auditctl -a always,exit -F arch=b32 -S execve -F path=/bin/dash
auditctl -a always,exit -F arch=b64 -S execve -F path=/bin/dash

These rules audit all process creations of the /bin/dash binary. /bin/sh doesn't work here, because it's a symlink and audit apparently only sees the target file name:

server ~ # ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Nov  8  2014 /bin/sh -> dash*

A simple test should now produce audit logs in /var/log/audit/audit.log (I've taken the liberty and added a lot of line breaks to improve the readability):

server ~ # sh -c 'echo test'
test

server ~ # tail -f /var/log/audit/audit.log
[...]
type=SYSCALL msg=audit(1488219335.976:43871): arch=40000003 syscall=11 \
  success=yes exit=0 a0=ffdca3ec a1=f7760e58 a2=ffdd399c a3=ffdca068 items=2 \
  ppid=27771 pid=27800 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 \
  fsgid=0 tty=pts7 ses=7532 comm="sh" exe="/bin/dash" key=(null)
type=EXECVE msg=audit(1488219335.976:43871): argc=3 a0="sh" a1="-c" \
  a2=6563686F2074657374
type=CWD msg=audit(1488219335.976:43871):  \
  cwd="/var/lib/iserv/remote-support/iserv-martin.von.wittich"
type=PATH msg=audit(1488219335.976:43871): item=0 name="/bin/sh" inode=10403900 \
  dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
type=PATH msg=audit(1488219335.976:43871): item=1 name=(null) inode=5345368 \
  dev=08:01 mode=0100755 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL
type=PROCTITLE msg=audit(1488219335.976:43871): \
  proctitle=7368002D63006563686F2074657374

Lots of the information is encoded, but ausearch can translate it with -i:

server ~ # ausearch -i -x /bin/dash | tail                                      
[...]
----
type=PROCTITLE msg=audit(27.02.2017 19:15:35.976:43871) : proctitle=sh 
type=PATH msg=audit(27.02.2017 19:15:35.976:43871) : item=1 name=(null) \
  inode=5345368 dev=08:01 mode=file,755 ouid=root ogid=root rdev=00:00 \
  nametype=NORMAL 
type=PATH msg=audit(27.02.2017 19:15:35.976:43871) : item=0 name=/bin/sh \
  inode=10403900 dev=08:01 mode=file,755 ouid=root ogid=root rdev=00:00 \
  nametype=NORMAL 
type=CWD msg=audit(27.02.2017 19:15:35.976:43871) :  \
  cwd=/var/lib/iserv/remote-support/iserv-martin.von.wittich 
type=EXECVE msg=audit(27.02.2017 19:15:35.976:43871) : argc=3 a0=sh a1=-c \
  a2=echo test 
type=SYSCALL msg=audit(27.02.2017 19:15:35.976:43871) : arch=i386 \
  syscall=execve success=yes exit=0 a0=0xffdca3ec a1=0xf7760e58 a2=0xffdd399c \
  a3=0xffdca068 items=2 ppid=27771 pid=27800 auid=root uid=root gid=root \
  euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts7 \
  ses=7532 comm=sh exe=/bin/dash key=(null) 
----

If you don't want to restrict the ausearch filtering to /bin/dash, you can also use ausearch -i -m ALL to translate the complete log. Another good filter would be ausearch -i -p <PID of a zombie process>, in this case ausearch -i -p 27800.

Just leave these rules in place until new zombie processes show up, and then search for the process creation of a zombie PID:

ausearch -i -p <PID>

This should be very helpful to identify the root cause of the zombie processes. In my case it was a PHP script that used proc_open to spawn a Perl script without closing the handle with proc_close.

Shell Pipe Tee – How to Reuse Pipe Data for Different Commands

You may use tee to duplicate command for processing whole stream by many command:

( ( seq 1 10 | tee /dev/fd/5 | sed s/^/line..\ / >&4 ) 5>&1 | wc -l ) 4>&1 
line.. 1
line.. 2
line.. 3
line.. 4
line.. 5
line.. 6
line.. 7
line.. 8
line.. 9
line.. 10
10

or split line by line, using bash:

while read line ;do
    echo cmd1 $line
    read line && echo cmd2 $line
    read line && echo cmd3 $line
  done < <(seq 1 10)
cmd1 1
cmd2 2
cmd3 3
cmd1 4
cmd2 5
cmd3 6
cmd1 7
cmd2 8
cmd3 9
cmd1 10

Finaly there is a way for running cmd1, cmd2 and cmd3 only once with 1/3 of stream as STDIN:

( ( ( seq 1 10 |
         tee /dev/fd/5 /dev/fd/6 |
           sed -ne '1{:a;p;N;N;N;s/^.*\n//;ta;}' |
           cmd1 >&4
     ) 5>&1 |
       sed -ne '2{:a;p;N;N;N;s/^.*\n//;ta;}' |
       cmd2 >&4
  ) 6>&1 |
    sed -ne '3{:a;p;N;N;N;s/^.*\n//;ta;}' |
    cmd3 >&4
) 4>&1 
command_1: 1
command_1: 4
command_1: 7
command_1: 10
Command-2: 2
Command-2: 5
Command-2: 8
command 3: 3
command 3: 6
command 3: 9

For trying this, you could use:

alias cmd1='sed -e "s/^/command_1: /"' \
    cmd2='sed -e "s/^/Command_2: /"' \
    cmd3='sed -e "s/^/Command_3: /"'

For using one stream on different process if on same script, you could do:

(
    for ((i=(RANDOM&7);i--;));do
        read line;
        echo CMD1 $line
      done
    for ((i=RANDOM&7;i--;));do
        read line
        echo CMD2 $line
      done
    while read line ;do
        echo CMD3 $line
      done
)
CMD1 1
CMD1 2
CMD1 3
CMD2 4
CMD2 5
CMD2 6
CMD2 7
CMD2 8
CMD2 9
CMD3 10

For this, you may have to transform your separated scripts into bash function to be able to build one overall script.

Another way could be to ensure each script won't output anything to STDOUT, than add a cat at end of each script to be able to chain them:

#!/bin/sh

for ((i=1;1<n;i++));do
   read line
   pRoCeSS the $line
   echo >output_log
 done

cat

Final command could look like:

seq 1 10 | cmd1 | cmd2 | cmd2

Best Answer

Related Solutions

Way to identify which process turns into Zombie process

Shell Pipe Tee – How to Reuse Pipe Data for Different Commands

Related Question