Shell – Hold multiple space containing arguments in a single variable

shellshell-scriptvariable

I was trying to debug my shell script:

content_type='-H "Content-Type: application/json"'
curl $content_type -d "{"test": "test"}" example.com

I learned that this does not do what I expect. In this script, the arguments being passed to curl (due to $content_type) are:

-H
"Content-Type:
application/json"

instead of (what I expected):

-H
Content-Type: application/json

I know that I can write the script as follows:

content_type="Content-Type: application/json"
curl -H "$content_type" -d "{"test": "test"}" example.com

and it would work this way but I am wondering if it is possible to hold multiple space containing arguments in a single variable.

With the second (working) version, if I want to exclude the content type, I need to remove two things: -H and $content_type.

However, I want to know if it is possible to put the option and its value into a single entity, so that excluding/including the content type will result in removing/adding a single entity.

Using arrays is not portable because there are no arrays in POSIX.

Why does the first method behave that way?

The reason is word splitting. When we define the content_type as follows:

content_type='-H "Content-Type: application/json"'

it has the value:

-H "Content-Type: application/json"

When we reference that variable without quotes (that is, $content_type, instead of "$content_type"), the expanded value becomes a subject of word splitting. From word splitting page of bash manual:

The shell scans the results of parameter expansion, command
substitution, and arithmetic expansion that did not occur within
double quotes for word splitting.

From the same page:

The shell treats each character of $IFS as a delimiter, and splits the
results of the other expansions into words using these characters as
field terminators. If IFS is unset, or its value is exactly
<space><tab><newline>, the default …

So, -H "Content-Type: application/json" is splitted by using <space><tab><newline>as delimiters. That gives us:

-H
"Content-Type:
application/json"

Best Answer

There is one standard array like structure in the standard shell. It is the positionnal parameters. So a possible solution is to use set -- … and "$@". In your case you would do:

set -- -H "Content-Type: application/json"
curl "$@" -d "{"test": "test"}" example.com

Note that this limit you to only one available array. You can't have one for curl and the other for another program for example. And of course it trashes the arguments to your script.

Related Solutions

Shell – Understanding IFS

Yes, they are the same.
Yes.
In bash, and similar shells, you could do something like IFS=$' \t\n'. Otherwise, you could insert the literal control codes by using [space] CTRL+V [tab] CTRL+V [enter]. If you are planning to do this, however, it's better to use another variable to temporarily store the old IFS value, and then restore it afterwards (or temporarily override it for one command by using the var=foo command syntax).
- The first code snippet will put the entire line read, verbatim, into $line, as there are no field separators to perform word splitting for. Bear in mind however that since many shells use cstrings to store strings, the first instance of a NUL may still cause the appearance of it being prematurely terminated.
- The second code snippet may not put an exact copy of the input into $line. For example, if there are multiple consecutive field separators, they will be made into a single instance of the first element. This is often recognised as loss of surrounding whitespace.
- The third code snippet will do the same as the second, except it will only split on a space (not the usual space, tab, or newline).

Bash – Why Does Bash Add Single Quotes to Unquoted Failed Pathname Expansions?

When instructed to echo commands as they are executed ("execution trace"), both bash and ksh add single quotes around any word with meta-characters (*, ?, ;, etc.) in it.

The meta-characters could have gotten into the word in a variety of ways. The word (or part of it) could have been quoted with single or double quotes, the characters could have been escaped with a \, or they remained as the result of a failed filename matching attempt. In all cases, the execution trace will contain single-quoted words, for example:

$ set -x
$ echo foo\;bar
+ echo 'foo;bar'

This is just an artifact of the way the shells implement the execution trace; it doesn't alter the way the arguments are ultimately passed to the command. The quotes are added, printed, and discarded. Here is the relevant part of the bash source code, print_cmd.c:

/* A function to print the words of a simple command when set -x is on. */
void
xtrace_print_word_list (list, xtflags)
...
{
  ...
  for (w = list; w; w = w->next)
    {
      t = w->word->word;
      ...
      else if (sh_contains_shell_metas (t))
        {
          x = sh_single_quote (t);
          fprintf (xtrace_fp, "%s%s", x, w->next ? " " : "");
          free (x);
        }

As to why the authors chose to do this, the code there doesn't say. But here's some similar code in variables.c, and it comes with a comment:

/* Print the value cell of VAR, a shell variable.  Do not print
   the name, nor leading/trailing newline.  If QUOTE is non-zero,
   and the value contains shell metacharacters, quote the value
   in such a way that it can be read back in. */
void
print_var_value (var, quote)
...
{
  ...
  else if (quote && sh_contains_shell_metas (value_cell (var)))
    {
      t = sh_single_quote (value_cell (var));
      printf ("%s", t);
      free (t);
    }

So possibly it's done so that it's easier to copy the command lines from the output of the execution trace and run them again.

Why does the first method behave that way?

Best Answer

Related Solutions

Shell – Understanding IFS

Bash – Why Does Bash Add Single Quotes to Unquoted Failed Pathname Expansions?

Related Question