I had a problem in a bash backup script caused by the following behavior that I don't understand.
First, we create a parameter holding a string with 2 consecutive spaces (right before '<'):
me@mysystem:~$ TEST="a string with 2 spaces right <--HERE!"
me@mysystem:~$ echo $TEST
a string with 2 spaces right <--HERE!
I can grep a space:
me@mysystem:~$ echo ${TEST} | grep " "
a string with 2 spaces right <--HERE!
I can grep a space followed by another letter:
me@mysystem:~$ echo ${TEST} | grep " s"
a string with 2 spaces right <--HERE!
I can grep the part after the 2 spaces:
me@mysystem:~$ echo ${TEST} | grep "HERE"
a string with 2 spaces right <--HERE!
I can do a variable expansion replacing both spaces with an underscore:
me@mysystem:~$ echo ${TEST// /_}
a_string_with_2_spaces_right__<--HERE!
But grepping the double space requires quoting the variable:
me@mysystem:~$ echo $TEST | grep " "; echo "Done."
Done.
me@mysystem:~$ echo ${TEST} | grep " "; echo "Done."
Done.
me@mysystem:~$ echo "$TEST" | grep " "; echo "Done."
a string with 2 spaces right <--HERE!
Done.
me@mysystem:~$ echo "${TEST}" | grep " "; echo "Done."
a string with 2 spaces right <--HERE!
Done.
I don't understand why this is the case.
Long story short: I had an if statement in the script checking whether ${FILE} represents an existing file, and that apparently also stops interpreting the variable when encountering a space unless quoted. So I'm not sure anymore what ${} does or doesn't actually take care off.
It's best to always quote the variable I guess?
Best Answer
This is a classic case of word splitting, done by shell, on unquoted variable expansion.
Basically when you do,
$TEST
(without quotes ("$TEST"
)), the shell:Performs word splitting, based on the value of
IFS
environment variable (default: space, tab, newline), and splits the whole expansion into separate wordsso as you have two consecutive spaces, the word splitting (and also pathname (glob) expansion if any of
*
,?
,[]
is present) is done, and when the output string is rebuilt the words are joined by space, as a result you have only one space betweenright
and<--HERE!
. This would be true for any combination ofIFS
characters.In a nutshell, quote your variable expansions: