Your quoting is wrong. When you write $CMD
with no quotes, the value of $CMD
is broken up into “words” at each whitespace sequence¹ (the words can contain any non-whitespace character including punctuation), and then each word undergoes globbing (i.e. wildcard expansion). Note that quotes in the value of CMD
, in particular, are untouched: quotes have a meaning in the syntax of shell scripts but not in variable substitution. After this, the first word becomes the name of the command to execute, and subsequent words are the command's arguments.
In your example, assuming $KEYNAME
is somekeyname
and $HZID
is somehzid
, then the
command and arguments are:
./dnscurl.pl
--keyname
$KEYNAME
--
-X
POST
-H
"Content-Type:
text/xml;
charset=UTF-8"
--upload-file
/tmp/file.xml
https://route53.amazonaws.com/2010-10-01/hostedzone/somehzid/rrset
Note that text/xml;
appears as the first non-option argument; clearly the Perl script passes that argument down to curl
. The --
in the argument list is unrelated to your problem.
There's no way to stuff a command line into a variable. A (simple) variable is the wrong tool for that: it contains a string, but a command line is a list of strings (the command, and its arguments). You can stuff a command name and its argument into an array variable:
CMD=(./dnscurl.pl --keyname "$KEYNAME" --
-X POST -H "Content-Type: text/xml; charset=UTF-8"
--upload-file /tmp/file.xml
"https://route53.amazonaws.com/2010-10-01/hostedzone/$HZID/rrset")
RESULT=$("${CMD[@]}")
The strange-looking syntax "${CMD[@]}"
expands the array variable CMD
to the list of words in the array. The double quotes prevent expansion of words inside the array, and [@]
is needed for historical reasons to tell the shell that you want to expand an array.
Another way to remember a command line, or an arbitrary shell snippet, for later use, is a function.
cmd () {
./dnscurl.pl --keyname "$KEYNAME" -- \
-X POST -H "Content-Type: text/xml; charset=UTF-8" \
--upload-file /tmp/file.xml \
"https://route53.amazonaws.com/2010-10-01/hostedzone/$HZID/rrset"
}
RESULT=$(cmd)
¹ More precisely, according to the value of IFS
.
Best Answer
The POSIX standard imposes word expansion to be done in the following order (emphasize is mine):
The only point which interests us here is the first one: as you can see tilde expansion is processed before parameter expansion:
echo $x
, there is no tilde to be found, so it proceeds.echo $x
,$x
is found and expanded and the command-line becomesecho ~/someDirectory
.~
character remains as-is.By using the quotes while assigning the
$x
, you were explicitly requesting to not expand the tilde and treat it like a normal character. A thing often missed is that in shell commands you don't have to quote the whole string, so you can make the expansion happen right during the variable assignment:And you can also make the expansion occur on the
echo
command-line as long as it can happen before parameter expansion:If for some reason you really need to affect the tilde to the
$x
variable without expansion, and be able to expand it at theecho
command, you must proceed in two times to force two expansions of the$x
variable to occur:However, be aware that depending on the context where you use such structure it may have unwanted side-effect. As a rule of thumb, prefer to avoid using anything requiring
eval
when you have another way.If you want to specifically address the tilde issue as opposed to any other kind of expansion, such structure would be safer and portable:
This structure explicitly check the presence of a leading
~
and replaces it with the user home dir if it is found.Following your comment, the
x="${HOME}/${x#"~/"}"
may indeed be surprising for someone not used in shell programming, but is in fact linked to the same POSIX rule I quoted above.As imposed by the POSIX standard, quote removal happens last and parameter expansion happens very early. Thus,
${#"~"}
is evaluated and expanded far before the evaluation of the outer quotes. In turns, as defined in Parameter expansion rules:Thus, the right side of the
#
operator must be properly quoted or escaped to avoid tilde expansion.So, to state it differently, when the shell interpretor looks at
x="${HOME}/${x#"~/"}"
, he sees:${HOME}
and${x#"~/"}
must be expanded.${HOME}
is expanded to the content of the$HOME
variable.${x#"~/"}
triggers a nested expansion:"~/"
is parsed but, being quoted, is treated as a literal1. You could have used single quotes here with the same result.${x#"~/"}
expression itself is now expanded, resulting in the prefix~/
being removed from the value of$x
.${HOME}
, the literal/
, the expansion${x#"~/"}
.a=$b
I usually find it clearer add double-quotes.By-the-way, if look more closely to the
case
syntax, you will see the"~/"*
construction which relies on the same concept asx=~/'someDirectory'
I explained above (here again, double and simple quotes could be used interchangeably).Don't worry if these things may seem obscure at the first sight (maybe even at the second or later sights!). In my opinion, parameter expansion are, with subshells, one of the most complex concept to grasp when programming in shell language.
I know that some people may vigorously disagree, but if you would-like to learn shell programming more in depth I encourage you to read the Advanced Bash Scripting Guide: it teaches Bash-scripting, so with a lot of extensions and bells-and-whistles compared to POSIX shell scripting, but I found it well written with loads of practical examples. Once you manage this, it is easy to restrict yourself to POSIX features when you need to, I personally think that entering directly in the POSIX realm is an unnecessary steep learning curve for beginners (compare my POSIX tilde replacement with @m0dular's regex-like Bash equivalent to get an idea of what I mean ;) !).
1: Which leads me into finding a bug in Dash which don't implement tilde expansion here correctly (verifiable using
x='~/foo'; echo "${x#~/}"
). Parameter expansion is a complex field both for the user and the shell developers themselves!