Bash – Using Shell Variables for Command Options

bashvariable

In a Bash script, I'm trying to store the options I'm using for rsync in a separate variable. This works fine for simple options (like --recursive), but I'm running into problems with --exclude='.*':

$ find source
source
source/.bar
source/foo

$ rsync -rnv --exclude='.*' source/ dest
sending incremental file list
foo

sent 57 bytes  received 19 bytes  152.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

$ RSYNC_OPTIONS="-rnv --exclude='.*'"

$ rsync $RSYNC_OPTIONS source/ dest
sending incremental file list
.bar
foo

sent 78 bytes  received 22 bytes  200.00 bytes/sec
total size is 0  speedup is 0.00 (DRY RUN)

As you can see, passing --exclude='.*' to rsync "manually" works fine (.bar isn't copied), it doesn't work when the options are stored in a variable first.

I'm guessing that this is either related to the quotes or the wildcard (or both), but I haven't been able to figure out what exactly is wrong.

Best Answer

In general, it's a bad idea to demote a list of separate items into a single string, no matter whether it's a list of command line options or a list of pathnames.

Using an array instead:

rsync_options=( -rnv --exclude='.*' )

rsync_options=( -r -n -v --exclude='.*' )

and later...

rsync "${rsync_options[@]}" source/ target

This way, the quoting of the individual options is maintained (as long as you double quote the expansion of ${rsync_options[@]}). It also allows you to easily manipulate the individual entries of the array, would you need to do so, before calling rsync.

In any POSIX shell, one may use the list of positional parameters for this:

set -- -rnv --exclude='.*'

rsync "$@" source/ target

Again, double quoting the expansion of $@ is critical here.

Tangentially related:

How can we run a command stored in a variable?

The issue is that when you put the two sets of option into a string, the single quotes of the --exclude option's value becomes part of that value. Hence,

RSYNC_OPTIONS='-rnv --exclude=.*'

would have worked¹... but it's better (as in safer) to use an array or the positional parameters with individually quoted entries. Doing so would also allow you to use things with spaces in them, if you would need to, and avoids having the shell perform filename generation (globbing) on the options.

^{¹ provided that $IFS is not modified and that there's no file whose name starts with --exclude=. in the current directory, and that the nullglob or failglob shell options are not set.}

Related Solutions

Shell Scripts – Naming Conventions for Variables

Environment variables or shell variables that are introduced by the operating system, shell startup scripts, or by the shell itself etc. are usually all in CAPITALS.

To prevent your own variables from conflicting with these variables, it is a good practice to use lower_case variable names.

Bash – Shell variables in a for loop

A nameref variable is to a “normal” variable what a symbolic link is to a regular file.

$ typeset -n ref=actual; ref=foo; echo "$actual"
foo

A for loop executes the body with the loop variable (the “control variable”) bound in turn to each word in the list.

$ for x in one two three; do echo "$x"; done
one
two
three

This is equivalent to writing out successive assignments:

x=one; echo "$x"
x=two; echo "$x"
x=three; echo "$x"

If the loop variable is a nameref, then the body is executed with the nameref targeting each element of the word list in turn. This is not equivalent to a series of assignment like above: an assignment ref=value where ref is a nameref would affect the variable that ref is pointing to, but the for loop changes where the nameref points rather than following the reference to change the variable that it points to.

$ original=0; one=1; two=2; three=3
$ typeset -n ref=original
$ echo $ref
0
$ for ref in one two three; do echo "$ref"; done
1
2
3
$ echo original
0

The indirection can be observed through assignments as well, if you assign to the loop variable (which is uncommon, but permitted).

$ one=1; two=2; three=3
$ typeset -n ref
$ for ref in one two three; do echo ref=$((ref+10)); done
$ echo "$one $two $three"
11 12 13

The last sentence explains that the target of a nameref can be an array. The nameref itself isn't an array, it's still a scalar variable, but when it's used in an assignment or in a dereference, it acts as the same type as the variable it points to.

$ a=(0 1 2)
$ typeset -n ref=a
$ ref[1]=changed
$ echo "${a[@]}"
0 changed 2

Best Answer

Related Solutions

Shell Scripts – Naming Conventions for Variables

Bash – Shell variables in a for loop

Related Question