Shell Script – Why ‘Set -e’ Doesn’t Work in Subshells with OR List

shellshell-script

I've run across some scripting like this recently:

( set -e ; do-stuff; do-more-stuff; ) || echo failed

This looks fine to me, but it does not work! The set -e does not apply, when you add the ||. Without that, it works fine:

$ ( set -e; false; echo passed; ); echo $?
1

However, if I add the ||, the set -e is ignored:

$ ( set -e; false; echo passed; ) || echo failed
passed

Using a real, separate shell works as expected:

$ sh -c 'set -e; false; echo passed;' || echo failed
failed

I've tried this in multiple different shells (bash, dash, ksh93) and all behave the same way, so it's not a bug. Can someone explain this?

Best Answer

According to this thread, it's the behavior POSIX specifies for using "set -e" in a subshell.

(I was surprised as well.)

First, the behavior:

The -e setting shall be ignored when executing the compound list following the while, until, if, or elif reserved word, a pipeline beginning with the ! reserved word, or any command of an AND-OR list other than the last.

The second post notes,

In summary, shouldn't set -e in (subshell code) operate independently of the surrounding context?

No. The POSIX description is clear that surrounding context affects whether set -e is ignored in a subshell.

There's a little more in the fourth post, also by Eric Blake,

Point 3 is not requiring subshells to override the contexts where set -e is ignored. That is, once you are in a context where -e is ignored, there is nothing you can do to get -e obeyed again, not even a subshell.
$ bash -c 'set -e; if (set -e; false; echo hi); then :; fi; echo $?' 
hi 
0 
Even though we called set -e twice (both in the parent and in the subshell), the fact that the subshell exists in a context where -e is ignored (the condition of an if statement), there is nothing we can do in the subshell to re-enable -e.

This behavior is definitely surprising. It is counter-intuitive: one would expect the re-enabling of set -e to have an effect, and that the surrounding context would not take precedent; further, the wording of the POSIX standard does not make this particularly clear. If you read it in the context where the command is failing, the rule does not apply: it only applies in the surrounding context, however, it applies to it completely.

Expansion by the shell

The quotes around "$i*.csv" make the difference. With the quotes, the shell expands that string to "2014-11/*.csv". That exact file doesn't exist, and zip reports an error. Without quotes, the * also expands (via filename expansion/"globbing"), and the resulting zip command is a complete list of matching files, each as a separate argument. You can get the second behaviour, inside the for loop, with:

for i in */ ; do zip "zips/${i%/}.zip" "$i"*.csv ; done

Expansion by zip

zip can also expand wildcards for itself, but not in all situations. From the zip manual:

The zip program can do the same matching on names that are in the zip archive being modified or, in the case of the -x (exclude) or -i (include) options, on the list of files to be operated on, by using backslashes or quotes to tell the shell not to do the name expansion.

The original command works on subsequent attempts, after you've successfully created an archive, because zip tries to match the wildcards against the contents of the existing archive. They exist there, and still exist on the filesystem, so they're reported with updating:.

To get zip to handle the wildcards when creating the archive, use the -r (recurse) option to recurse into the requested directory, and -i (include) to limit it to files matching the pattern:

 for i in */ ; do zip -r "zips/${i%/}.zip" "$i" -i '*.csv' ; done

Best Answer

Related Solutions

Bash – Why does this “while read” work in a terminal, but not in a shell script

Shell – Why does `zip` in a for loop work when the file exists, but not when it doesn’t

Expansion by the shell

Expansion by zip

Related Question