Process not closing when stdin is closed

pipeprocess

I am starting a process with a program. I expect the process to terminate when the program does, since it loses its stdin.

I terminated the program, then went to the proc/pid/fd for the process, and found that its stdin is still linked to /dev/pts/2.

Why won't the process close in this case? Better yet, is there a wrapper or technique I can use to ensure that the program will close when its stdin pipe closes?

Best Answer

stdin is file descriptor 0. Closing a file descriptor of a process is something that can only be done actively by the process itself. stdin is closed when the process decides to close it period.

Now, when the stdin of a process is the reading end of a pipe, the other end of the pipe can be open by one or more other processes. When all the file descriptors to the other end have been closed, reading from that pipe will read the remaining data still in that pipe, but will then end up returning nothing (instead of waiting for more data) meaning end-of-file.

Applications like cat, cut, wc... that read from their stdin will usually exit when that happens because their role is to process their input till the end until there's no more input.

There's no magic mechanism that causes applications to die when the end of their input is reached, only them deciding to exit when that happens.

In:

echo foo | cat

Once echo has written "foo\n", it exits which causes the writing end of the pipe to be closed, then the read() done by cat at the other end returns 0 bytes, which tells cat there's nothing more to read and then cat decides to exit.

In

echo foo | sleep 1

sleep only exits after 1 second has elapsed. Its stdin becoming a closed pipe has no incidence on that, sleep is not even reading from its stdin.

It's different on the writing end of pipes (or sockets for that matters).

When all the fds on the reading end have been closed, any attempt to write on the fds open to the writing end causes a SIGPIPE to be sent to the process causing it to die (unless it ignores the signal in which case the write() fails with EPIPE).

But that only happens when they try to write.

For instance, in:

sleep 1 | true

Even though true exits straight away and the reading end is then closed straight away, sleep is not killed because it doesn't attempt to write to its stdout.

Now, about /proc/fd/pid/n showing in red in the ls -l --color output (as mentioned in the first version of your question), that's only because ls does a lstat() on the result of readlink() on that symlink to try and determine the type of the target of the link.

For file descriptors opened on pipes, or sockets or files in other namespaces, or deleted files, the result of readlink will not be an actual path on the file system, so the second lstat() done by ls will fail and ls will think it's a broken symlink, and broken symlinks are rendered in red. You'll get that with any fd to any end of any pipe, whether the other end of the pipe is closed or not. Try with ls --color=always -l /proc/self/fd | cat for instance.

To determine whether a fd points to a broken pipe, on Linux, you can try lsof with the -E option.

$ exec 3> >(:) 4> >(sleep 999)
$ lsof -ad3-4 -Ep "$$"
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
zsh     13155 stephane    3w  FIFO   0,10      0t0 5322414 pipe
zsh     13155 stephane    4w  FIFO   0,10      0t0 5323312 pipe 392,sleep,0r

For the fd 3, lsof was not able to find any other process at the reading end of the pipe. Beware though, that you could get output like:

$ exec 5<&3
$ lsof -ad3-5 -Ep "$$"
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
zsh     13155 stephane    3w  FIFO   0,10      0t0 5322414 pipe 13155,zsh,5w
zsh     13155 stephane    4w  FIFO   0,10      0t0 5323312 pipe 392,sleep,0r
zsh     13155 stephane    5w  FIFO   0,10      0t0 5322414 pipe 392,sleep,3w 13155,zsh,3w

fds 3 and 5 are still to broken pipes, because there's no fd to the reading end (there seems to be a bug in lsof, since the fact that sleep also has its fd 3 open to the broken pipe is not reflected everywhere).

To kill a process as soon as the pipe open on its stdin loses its last writer (becomes broken), you could do something like:

run_under_watch() {
  perl -MIO::Poll -e '
     if ($pid = fork) {
       $SIG{CHLD} = sub {
         wait;
         exit($? & 127 ? ($? & 127) + 128 : $? >> 8);
       };
       $p = IO::Poll->new; $p->mask(STDIN, POLLERR); $p->poll;
       kill "TERM", $pid;
       sleep 1;
       kill "KILL", $pid;
       exit(1);
     } else {
       exec @ARGV
     }' "$@"
 }

Which would watch for an error condition on stdin (and on Linux, that seems to happen as soon as there's no writer left, even if there's data left in the pipe) and kill the child command as soon as it happens. For instance:

 sleep 1 | run_under_watch sleep 2

Would kill the sleep 2 process after 1 second.

Now, generally that's a bit of a silly thing to do. That means you're potentially killing a command before it has had time to process the end of its input. For instance, in:

 echo test | run_under_watch cat

You'll find that cat is sometimes killed before it has had time to output (or even to read!) "test\n". There's no way around that, our watcher can't know how much time the command needs to process the input. All we can do is give a grace period before the kill "TERM" hoping it's enough for the command to read the content left in the pipe and do what it needs to do with it.

Related Question