The ! character triggers history expansion, as you discovered. This step occurs before a command is saved to the history, so that it can saved into the history with the expansion already done. When an error occurs in history expansion, bash stops processing the command, and so it never gets saved into the history.
History expansion allows previous commands or parts of previous commands to be substituted into the current command. If this was done before saving the command to the history, it could mean something different each time it is executed, because the previous commands are different in each instance. See http://www.gnu.org/software/bash/manual/bashref.html#History-Interaction
For reference, zsh's behavior is different. It will save the command but with the history expansion missing.
This is happening because cut is outputting NULL characters in the output. You can't pass a program arguments which contain a null character (see this).
In bash this works because bash can't handle NULL characters in strings, and it strips them out. Zsh is a bit more powerful, and it can handle NULL characters. However when it comes time to pass the string to the program, it still contains the null, which signals the end of the argument.
Let's look at this in detail.
$ echo 'English/Flavors/AU/stuff1/filename.txt' | cut -d'/' --output-delimiter '' -f1,3 | xxd
0000000: 456e 676c 6973 6800 4155 0a English.AU.
Here we simulated one of your files, passing the path through cut. Notice the xxd output which has a NULL character between English and AU.
Now lets run through and simulate the rest of the script.
$ l=$(echo 'English/Flavors/AU/stuff1/filename.txt' | cut -d'/' --output-delimiter '' -f1,3)
$ dest=/stuff2/${l}.utf8.txt
$ echo "$dest" | xxd
0000000: 2f73 7475 6666 322f 456e 676c 6973 6800 /stuff2/English.
0000010: 4155 2e75 7466 382e 7478 740a AU.utf8.txt.
Notice the NULL after the English. The echo displays it properly because echo is a shell built-in. If we use an external echo, it also exhibits the issue.
$ /bin/echo "$dest" | xxd
0000000: 2f73 7475 6666 322f 456e 676c 6973 680a /stuff2/English.
P.S. You really should be quoting too :-)
The solution is to not use cut, use awk instead.
$ echo 'English/Flavors/AU/stuff1/filename.txt' | awk -F/ '{ print $1$3 }' | xxd
0000000: 456e 676c 6973 6841 550a EnglishAU.
Best Answer
In string:
you have
?in that string, so the shell will perform pathname expansion on that string, using pattern matching rules.In
bash, iffailgloboptions was not set, which is default, then failed pattern will be left as-is:While
zshwill report no pattern match error withnomatchoption set, which is default:You can make
zshsuppress the error and print the pattern:You can make
bashbehave likezshwithnomatchoption set, by turning onfailglob:More general, you can disable shell filename generation:
(or
set -o noglob,set +o noglob)or using one of shell quoting methods to make the shell treats
?and other pattern matching special characters literally.zshalso provide thenoglobbuiltin, which disable filename generation in any words for the following simple command: