Bash – Would someone provide an example of a Bash synchronous vs asynchronous job

bash

According to the Bash 4.4 changelog:

https://lists.gnu.org/archive/html/info-gnu/2016-09/msg00008.html

There are a few incompatible changes between bash-4.3 and bash-4.4.
Bash now retains the exit status only of asynchronous jobs, as opposed
to all jobs. This means that it is not possible to use `wait' to
retrieve the status of a previously-completed synchronous command.

https://fossies.org/diffs/bash/4.3.46_vs_4.4/CHANGES-diff.html

Bash only adds asynchronous commands to the table of background pids
whose status it remembers, to avoid it growing too large during
scripts that create and reap large numbers of child processes. This
means that `wait' no longer works on synchronous jobs, but $? can be
used to get the exit status in those cases.

I've already been bitten by some other breaking changes between 4.3 and 4.4 but I don't know how to write an example which tests this particular change.

What is the difference between a synchronous job versus an asynchronous one in Bash and where did it store a table of pids to be queried by wait?

Best Answer

Example:

4.3

bash-4.3$ (echo "$BASHPID"; exit 123)
5358
bash-4.3$ wait 5358; echo "$?"
123

4.4

bash-4.4$ (echo "$BASHPID"; exit 123)
12171
bash-4.4$ wait 12171
bash: wait: pid 12171 is not a child of this shell

You can no longer use wait to get the exit status of that subshell that run in foreground (synchronously). foreground refers to jobs of interactive shells, but the same applies for commands run in non-interactive shells.

Note that it also applies to background jobs that are later put in foreground with fg:

bash-4.4$ (sleep 10; exit 123) &
[1] 12857
bash-4.4$ fg
( sleep 10; exit 123 )
bash-4.4$ wait 12857
bash: wait: pid 12857 is not a child of this shell

With bash-4.3 and before bash would remember the exit status of every past background and foreground command. That would not be useful for foreground commands as usually, you don't know their pid in the script and also in things like:

cmd1 &
...
cmd2
wait "$!"

cmd1's pid could very well have been reused for cmd2. In that case wait "$!" would get you the exit status of cmd2 instead of cmd1. Recording the pid of only asynchronous commands slightly reduces the risk of wait giving you the exit status of the wrong command (beside the performance issue mentioned by @Christopher).

Related Solutions

Bash Scripting – Does Bash Support Using Pointers?

A pointer (to a location of memory) is not really a useful concept in anything higher-level than C, be it something like Python or the shell. References to objects are of course useful in high-level languages, perhaps even necessary for building complex data structures. But in most cases thinking in terms of memory addresses is too low level to be very useful.

In Bash (and other shells), you can get the values of array elements with the ${array[index]} notation, assign them with array[index]=... and get the number of elements in the array with ${#array[@]}. The expression inside the brackets is an arithmetic expression. As a made-up example, we could add a constant prefix to all array members:

for ((i=0 ; i < ${#array[@]} ; i++ )) ; do
    array[i]="foo-${array[i]}"
done

(If we only cared about the values, and not the indexes, just for x in "${array[@]}" ; do... would be fine.)

With associative or sparse arrays, a numerical loop doesn't make much sense, but instead we'd need to fetch the array keys/indexes with ${!array[@]}. E.g.

declare -A assoc=([foo]="123" [bar]="456")
for i in "${!assoc[@]}" ; do 
    echo "${assoc[$i]}"
done

In addition to that, Bash has two ways to point indirectly to another variable:

indirect expansion, using the ${!var} syntax, which uses the value of the variable whose name is in var, and
namerefs, which need to be created with the declare builtin (or the ksh-compatible synonym, typeset). declare -n ref=var makes ref a reference to the variable var.

Namerefs also support indexing, in that if we have arr=(a b c); declare -n ref=arr; then ${ref[1]} will expand to b. Using ${!p[1]} would instead take p as an array, and refer to the variable named by its second element.

In Bash, namerefs are literally that, references by name, and using a nameref from inside a function will use the local value of the named variable. This will print local value of var.

#!/bin/bash
fun() {
        local var="local value of var"
        echo "$ref";
}
var="global var"
declare -n ref=var
fun

BashFAQ has a longer article on indirection, too.

Best Answer

4.3

4.4

Related Solutions

Bash Scripting – Does Bash Support Using Pointers?

Related Question