Bash – Running piped bash script in background

background-processbashdaemonpipe

I'm attempting to build a monitoring script to watch localhost communication using netcat. I have two scripts that I've built, one to start the monitoring loop and one for the loop itself. They are as follows :

start.sh

#!/bin/bash
netcat localhost 1099  | bash loop.sh &

loop.sh

#!/bin/bash
while read sInput; do
     ...do something with $sInput
done

Simply running this application without in the foreground works fine but as soon as I try to run this script in the background it goes from running to stopped instantly. Could someone please educate me as to why this is happening and how to alleviate the problem.

My end goal is to have a bash script that I can create an arch linux daemon script with and have everything work perfectly. Your help is appreciated.

Best Answer

Sounds like it's reading from standard input (stdin). Try adding the -d (Do not attempt to read from stdin) option to netcat. Or redirect on the command line:

netcat localhost 1099 <&- | bash loop.sh &

You may also want to use nohup to make sure that it won't be adversely affected when/if you exit that shell before it stops.

Related Solutions

Bash – redirecting output of running background job in bash

Bash can't modify the file descriptors of a running process.

See answers for How to change the output redirection of a running process? (or a similar thread on stackoverflow)

The easiest and only current tool seems to be reredirect:

reredirect is a utility for taking an existing running program and attaching its outputs (standard output and error output) to files or another process. (quoting project homepage)

reredirect -m /dev/null <PID>

Bash – Notification Listener Behaving Strangely when Backgrounded

Newer derivatives of the OpenBSD netcat, including FreeBSD[1] and Debian[2], support a -d flag which prevents reading from stdin and fixes the problem you described.

The problem is that netcat is polling stdin as well as its "network" fd, and stdin is reopened from /dev/null in the second case above, where the shell function is run in the background before the pipeline is created. That means an immediate EOF on the first read from stdin (fd 0), but netcat will continue to poll(2) on the now-closed stdin, creating an endless loop.

Here is the redirection of stdin before the pipeline creation:

249 [pid 23186] open("/dev/null", O_RDONLY <unfinished ...>
251 [pid 23186] <... open resumed> )        = 3
253 [pid 23186] dup2(3, 0)                  = 0
254 [pid 23186] close(3)                    = 0

Now when netcat (pid 23187) calls its first poll(2), it reads EOF from stdin and closes fd 0:

444 [pid 23187] poll([{fd=4, events=POLLIN}, {fd=0, events=POLLIN}], 2, 4294967295) = 2 ([{fd=4, revents=POLLIN|POLLHUP}, {fd=0, revents=POLLIN}])
448 [pid 23187] read(0,  <unfinished ...>
450 [pid 23187] <... read resumed> "", 2048) = 0
456 [pid 23187] close(0 <unfinished ...>
458 [pid 23187] <... close resumed> )       = 0

The next call to accept(2) yields a client on fd 0, which is now the lowest-numbered free fd:

476 [pid 23187] accept(3,  <unfinished ...>
929 [pid 23187] <... accept resumed> {sa_family=AF_LOCAL, NULL}, [2]) = 0

Note here that netcat is now including fd 0 in the args to poll(2) twice: once for STDIN_FILENO, which is always included in the absence of the -d command-line parameter, and once for the newly-connected client:

930 [pid 23187] poll([{fd=0, events=POLLIN}, {fd=0, events=POLLIN}], 2, 4294967295) = 2 ([{fd=0, revents=POLLIN|POLLHUP}, {fd=0, revents=POLLIN|POLLHUP}])

The client sends EOF and netcat disconnects:

936 [pid 23187] read(0,  <unfinished ...>
938 [pid 23187] <... read resumed> "", 2048) = 0
940 [pid 23187] shutdown(0, SHUT_WR <unfinished ...>
942 [pid 23187] <... shutdown resumed> )    = 0
944 [pid 23187] close(0 <unfinished ...>
947 [pid 23187] <... close resumed> )       = 0

But now it's in trouble because it will continue to poll on fd 0, which is now closed. The netcat code does not handle the case of POLLNVAL being set in the .revents member of struct pollfd, so it gets into an endless loop, never to call accept(2) again:

949 [pid 23187] poll([{fd=0, events=POLLIN}, {fd=-1}], 2, 4294967295 <unfinished ...>
951 [pid 23187] <... poll resumed> )        = 1 ([{fd=0, revents=POLLNVAL}])
953 [pid 23187] poll([{fd=0, events=POLLIN}, {fd=-1}], 2, 4294967295 <unfinished ...>
955 [pid 23187] <... poll resumed> )        = 1 ([{fd=0, revents=POLLNVAL}])
...

In the first command, where the pipeline is backgrounded but is not run in a shell function, stdin is left open, so this case doesn't arise.

Code references (see the readwrite function):

Best Answer

Related Solutions

Bash – redirecting output of running background job in bash

Bash – Notification Listener Behaving Strangely when Backgrounded

Related Question