Bash – Putting Subshell in Background vs Putting Command in Background

bashforkshell-scriptsubshell

I have two bash scripts that try to check hosts that are up:

Script 1:

#!/bin/bash 
for ip in {1..254}; do
    ping -c 1 192.168.1.$ip | grep "bytes from" | cut -d" " -f 4 | cut -d ":" -f 1 &
done

Script 2:

#!/bin/bash 
for ip in {1..254}; do
    host=192.168.1.$ip
    (ping -c 1 $host > /dev/null 
    if [ "$?" = 0 ]
    then 
        echo $host
    fi) &
done

As I am checking a large range, I would like to process each ping command in parallel. However, my second script seems to not retry failed fork attempts due to resource limits. This results in the second script having inconsistent results while my first script gives constant results despite both failing to fork at times. Can someone explain this to me? Also is there anyway to retry failed forks?

Best Answer

There is already an answer which gives an improved code snippet to the task the original poster questions was related to, while it might not yet have more directly responded to the question.

The question is about differences of

  • A) Backgrounding a "command" directly, vs
  • B) Putting a subshell into the background (i.e with a similar task)

Lets check about those differences running 2 tests

# A) Backgrounding a command directly
sleep 2 & ps

outputs

[1] 4228
  PID TTY          TIME CMD
 4216 pts/8    00:00:00 sh
 4228 pts/8    00:00:00 sleep

while

# A) backgrounding a subhell (with similar tas)
( sleep 2; ) & ps

outputs something like:

[1] 3252
  PID TTY          TIME CMD
 3216 pts/8    00:00:00 sh
 3252 pts/8    00:00:00 sh
 3253 pts/8    00:00:00 ps
 3254 pts/8    00:00:00 sleep

** Test results:**

In this test (which run only a sleep 2) the subshell version indeed differs, as it would use 2 child processes (i.e. two fork()/exec operations and PID) and hence more than the direct backgrounding of the command.

In the script 1 of the question however the command was not a single sleep 2s but instead it was a pipe of 4 commands, which if we test in an additional case

  • C) Backgrounding a pipe with 4 commands
# C) Backgrounding a pipe with 4 commands
sleep 2s | sleep 2s | sleep 2s | sleep 2s & ps

yields this

[2] 3265
  PID TTY          TIME CMD
 3216 pts/8    00:00:00 bash
 3262 pts/8    00:00:00 sleep
 3263 pts/8    00:00:00 sleep
 3264 pts/8    00:00:00 sleep
 3265 pts/8    00:00:00 sleep
 3266 pts/8    00:00:00 ps

and shows that indeed the script 1 would be a much higher strain in terms of PIDs and fork()s.

As a rough estimate the script one would have used about 254 * 4 ~= 1000 PIDs and hence even more than the script 2 with 254 * 2 ~= 500 PIDs. Any problem occurring because of PIDs resouce depletion seems yet unlikely since at most Linux boxes 

$ cat /proc/sys/kernel/pid_max
32768

gives you 32x times the PIDs needed even for case script 1 and the processes/programs involved (i.e. sed , ping, etc) also seem unlikely to cause the inconstant results.

As mentioned by user @derobert the real issue behind the scripts failing was that the missing of the wait command, which means that after backgrounding the commands in the loop the end of the script and hence the shell caused all the child processes to be terminated.

Related Question