Shell Script – Should Variables Be Quoted When Executed?

quotingshell-scriptvariable

The general rule in shell scripting is that variables should always be quoted unless there is a compelling reason not to. For more details than you probably want to know, have a look at this great Q&A: Security implications of forgetting to quote a variable in bash/POSIX shells.

Consider, however, a function like the following:

run_this(){
    $@
}

Should $@ be quoted there or not? I played with it for a bit and couldn't find any case where the lack of quotes caused a problem. On the other hand, using the quotes makes it break when passing a command containing spaces as a quoted variable:

#!/usr/bin/sh
set -x
run_this(){
    $@
}
run_that(){
    "$@"
}
comm="ls -l"
run_this "$comm"
run_that "$comm"

Running the script above returns:

$ a.sh
+ comm='ls -l'
+ run_this 'ls -l'
+ ls -l
total 8
-rw-r--r-- 1 terdon users  0 Dec 22 12:58 da
-rw-r--r-- 1 terdon users 45 Dec 22 13:33 file
-rw-r--r-- 1 terdon users 43 Dec 22 12:38 file~
+ run_that 'ls -l'
+ 'ls -l'
/home/terdon/scripts/a.sh: line 7: ls -l: command not found

I can get around that if I use run_that $comm instead of run_that "$comm", but since the run_this (unquoted) function works with both, it seems like the safer bet.

So, in the specific case of using $@ in a function whose job is to execute $@ as a command, should $@ be quoted? Please explain why it should/shouldn't be quoted and give an example of data that can break it.

Best Answer

The problem lies in how the command is passed to the function:

$ run_this ls -l Untitled\ Document.pdf 
ls: cannot access Untitled: No such file or directory
ls: cannot access Document.pdf: No such file or directory
$ run_that ls -l Untitled\ Document.pdf 
-rw------- 1 muru muru 33879 Dec 20 11:09 Untitled Document.pdf

"$@" should be used in the general case where your run_this function is prefixed to a normally written command. run_this leads to quoting hell:

$ run_this 'ls -l Untitled\ Document.pdf'
ls: cannot access Untitled\: No such file or directory
ls: cannot access Document.pdf: No such file or directory
$ run_this 'ls -l "Untitled\ Document.pdf"'
ls: cannot access "Untitled\: No such file or directory
ls: cannot access Document.pdf": No such file or directory
$ run_this 'ls -l Untitled Document.pdf'
ls: cannot access Untitled: No such file or directory
ls: cannot access Document.pdf: No such file or directory
$ run_this 'ls -l' 'Untitled Document.pdf'
ls: cannot access Untitled: No such file or directory
ls: cannot access Document.pdf: No such file or directory

I'm not sure how I should pass a filename with spaces to run_this.

Related Solutions

Shell – Quoting and escaping

You should use like this:

"\"Test Internal Cluster\""

or wrap it in single quote:

'"Test Internal Cluster"'

Test:

#!/bin/bash

echo $1

Output:

% bash test.sh "\"Test Internal Cluster\""
"Test Internal Cluster"

Update

The problem here is becasue you call perl with $1, $1 contains white space, so perl will see it as three separate arguments.

To prevent this, you should wrap $1 in double quotes "$1".

Another note is you should handle $1 argument in perl script, because if $1 contains any special characters, it will cause some bugs in perl program. you can use quotemeta($ARGV[0]).

Note

I think you shoul not pass all argument as a string to shell script. You should pass them as normal, except cluster name, you still have to wrap it as string. Then pass them to perl script by "$@" instead of $1.

Bash – Why Does Bash Add Single Quotes to Unquoted Failed Pathname Expansions?

When instructed to echo commands as they are executed ("execution trace"), both bash and ksh add single quotes around any word with meta-characters (*, ?, ;, etc.) in it.

The meta-characters could have gotten into the word in a variety of ways. The word (or part of it) could have been quoted with single or double quotes, the characters could have been escaped with a \, or they remained as the result of a failed filename matching attempt. In all cases, the execution trace will contain single-quoted words, for example:

$ set -x
$ echo foo\;bar
+ echo 'foo;bar'

This is just an artifact of the way the shells implement the execution trace; it doesn't alter the way the arguments are ultimately passed to the command. The quotes are added, printed, and discarded. Here is the relevant part of the bash source code, print_cmd.c:

/* A function to print the words of a simple command when set -x is on. */
void
xtrace_print_word_list (list, xtflags)
...
{
  ...
  for (w = list; w; w = w->next)
    {
      t = w->word->word;
      ...
      else if (sh_contains_shell_metas (t))
        {
          x = sh_single_quote (t);
          fprintf (xtrace_fp, "%s%s", x, w->next ? " " : "");
          free (x);
        }

As to why the authors chose to do this, the code there doesn't say. But here's some similar code in variables.c, and it comes with a comment:

/* Print the value cell of VAR, a shell variable.  Do not print
   the name, nor leading/trailing newline.  If QUOTE is non-zero,
   and the value contains shell metacharacters, quote the value
   in such a way that it can be read back in. */
void
print_var_value (var, quote)
...
{
  ...
  else if (quote && sh_contains_shell_metas (value_cell (var)))
    {
      t = sh_single_quote (value_cell (var));
      printf ("%s", t);
      free (t);
    }

So possibly it's done so that it's easier to copy the command lines from the output of the execution trace and run them again.

Best Answer

Related Solutions

Shell – Quoting and escaping

Bash – Why Does Bash Add Single Quotes to Unquoted Failed Pathname Expansions?

Related Question