Bash – brace expansion and * expansion

bashbrace-expansionwildcards

Assume in the working directory, there is only one djvu file. I would like to backup the file into a file with file name having extra .bk at the end. cp *.djvu{,.bk} copies the djvu file into a file named *.djvu.bk. Why is * in the backup file name not expanded? How can it be?

Best Answer

What happens is that bash first expands *.djvu{,.bk} into *.djvu *.djvu.bk, and then does glob-expansion on those. This would explain what you observe: in your case, *.djvu, matches an existing file, say foo.djvu and expands into that, but *.djvu.bk matches no file, and thus expands as itself, *.djvu.bk.

The order of expansion is specified in the bash documentation:

The order of expansions is: brace expansion, tilde  expansion,  parame‐
ter,  variable  and arithmetic expansion and command substitution (done
in a left-to-right fashion), word splitting, and pathname expansion.

I would suggest rewriting your copy command as:

for f in *.djvu; do cp -- "$f" "$f".bk; done

Or perhaps, to avoid the syntactic overhead of an explicit for loop:

parallel -j1 cp -- {} {}.bk ::: *.djvu

(On second thoughts... that's not really much shorter.)

To answer your sub-question "how could it be expanded", one could use a sub-command (example in a directory containing just foo.djvu and bar.djvu):

$ echo $(echo *.djvu){,.bk}
bar.djvu foo.djvu bar.djvu foo.djvu.bk

But that isn't as safe a solution as the for loop or parallel call above; it will break down on file names containing white space.

Related Solutions

Bash – Deduplicating Bash Brace Expansions

Yes,

for f in *{b,c}*

is first expanded to:

for f in *b* *c*

And then the loop runs over the independent expansion of those two globs.

What you want here is one glob. Here, *[bc]* would do, but for anything more complex, in bash, you'd need to enable ksh-like extended globs. You'll probably want the nullglob option as well:

shopt -s nullglob extglob
for f in *@(b|c)*; do...

In zsh:

for f in *(b|c)*(N); do

The (N) being for a per-glob nullglob.

In ksh93:

for f in ~(N)*@(b|c)*; do

Matching files using curly brace expansion in zsh

You don't get a warning in bash, you get an error by ls (you'll find that ls exit status is non-0 which indicates it has failed).

In both zsh and bash, {...} is not a globbing operator, it's an expansion that occurs before globbing.

In:

ls -d -- *.{dot,svg,err}

(you forgot the -d and -- btw), the shell expands the {...} first:

ls -d -- *.dot *.svg *.err

and then does the glob. bash like most Bourne-like shells has that misfeature that non-matching globs are passed as-is. While on zsh, a non-matching glob is an error.

See how rm -f [ab].c in bash could delete the [ab].c file if there was no file called a.c nor b.c. In zsh, you'd get a no match error instead. See the failglob option in bash to get a similar behaviour.

ls -d -- *.{dot,svg,err}(N)

in zsh would enable the nullglob option on all 3 globs, so that if the globs don't match, they are removed, but that's probably not what you want, because if none of the globs match any file, the command will become:

ls -d --

Which will list . (the current directory) instead.

Best here is to use one glob that matches files with either of the 3 extensions:

ls -d -- *.(dot|svg|err)

That will give a sorted list of files to ls, ls will be run unless there's no file found matching that one pattern.

You also have the option to enable the sh/bash (bogus IMO) behaviour with emulate sh or with unsetopt nomatch. A slightly better approach is to enable the csh behaviour (which was also the behaviour of Unix shells before the Bourne shell was released):

setopt cshnullglob

With that option, the command is cancelled only if all the globs on the command line fail to match. If at least one matches, all the ones that don't match are removed so:

ls -d -- *.{dot,svg,err}

Will expand the dot, svg and err in turn, omitting the missing ones.

If you want to compare the effect on the order of the arguments of the different approaches, you need a command that (contrary to ls) doesn't sort them before displaying. With GNU ls, you can pass the -U option for that, or since ls does only print its arguments here, just use printf '%s\n' *... instead.

Best Answer

Related Solutions

Bash – Deduplicating Bash Brace Expansions

Matching files using curly brace expansion in zsh

Related Question