Bash – Why does bash not assign a fd to var when using external_command {var}>somefile

bashio-redirection

According to the manpage of bash, "redirection operators may precede or appear anywhere within
a simple command or may follow a command."

Also according to the man page, "A simple command is a sequence of optional variable assignments
followed by blank-separated words and redirections, and terminated by
a control operator."

Now here is the problem, what exactly does the shell mean by a simple command and a command?
Because, /bin/echo foo {var}> somefile results in var not being assigned a fd.
In contrast to that, echo foo {var}> somefile does result in a fd being assigned to var.
It appears that, unlike with the command, it works with the builtin. I have noted that this kind of construction can appear anywhere in a command that uses a builtin: {var}> someFile echo foo works. In this context, if a simple command consists of builtins, therefore a word or a token is also a builtin? Because that's what the manpage says: "Word: A sequence of characters considered as a single unit by the
shell. Also known as a token.""

I have also observed that fds get assigned when used with builtins and not external commands.
So what is the problem here? Is my observation correct? And what exactly is the difference between a simple command and a command. The manpage is very vague about this.

Best Answer

From the top of my head, I'd guess it's because when running an external command, the shell 1) forks, 2) handles redirections in the child process, and 3) exec's the actual command. With the assignment to var done in 2, in the child, it's not visible to the parent shell after the launched program exits. With a builtin command, there's no fork, and the shell juggles the fd's as necessary in the main shell process, and the variable assignment takes effect there.

Not that that matters much, since the redirection in some external command {var}>/whatever is useless anyway. The external command can't know what fd was opened for it, and while it could determine what fd's it has, there might be others than the one opened for the redirection on this line, so it can't reliably use that fd to output to /whatever. Instead, you'd usually use a fixed fd number, or have some command line argument or environment variable to tell the fd number to use.

But you can't do that here either, since the variable isn't yet set when the expansions on the command line are processed so it's quite hard to pass it to the started program. unset var; /bin/echo "var=$var" {var}>/dev/null outputs just var= and so does unset var; var=$var /usr/bin/env {var}>/dev/null |grep ^var. (Though in ksh and zsh the latter seems to pass an actual number through the environment.)

The only place where that kind of redirection seems to make sense, is exec {var}>/whatever, and that being a builtin, the variable is set in the main shell, and the value is there for the following commands to use.

Related Solutions

Bash – Why Does Bash Add Single Quotes to Unquoted Failed Pathname Expansions?

When instructed to echo commands as they are executed ("execution trace"), both bash and ksh add single quotes around any word with meta-characters (*, ?, ;, etc.) in it.

The meta-characters could have gotten into the word in a variety of ways. The word (or part of it) could have been quoted with single or double quotes, the characters could have been escaped with a \, or they remained as the result of a failed filename matching attempt. In all cases, the execution trace will contain single-quoted words, for example:

$ set -x
$ echo foo\;bar
+ echo 'foo;bar'

This is just an artifact of the way the shells implement the execution trace; it doesn't alter the way the arguments are ultimately passed to the command. The quotes are added, printed, and discarded. Here is the relevant part of the bash source code, print_cmd.c:

/* A function to print the words of a simple command when set -x is on. */
void
xtrace_print_word_list (list, xtflags)
...
{
  ...
  for (w = list; w; w = w->next)
    {
      t = w->word->word;
      ...
      else if (sh_contains_shell_metas (t))
        {
          x = sh_single_quote (t);
          fprintf (xtrace_fp, "%s%s", x, w->next ? " " : "");
          free (x);
        }

As to why the authors chose to do this, the code there doesn't say. But here's some similar code in variables.c, and it comes with a comment:

/* Print the value cell of VAR, a shell variable.  Do not print
   the name, nor leading/trailing newline.  If QUOTE is non-zero,
   and the value contains shell metacharacters, quote the value
   in such a way that it can be read back in. */
void
print_var_value (var, quote)
...
{
  ...
  else if (quote && sh_contains_shell_metas (value_cell (var)))
    {
      t = sh_single_quote (value_cell (var));
      printf ("%s", t);
      free (t);
    }

So possibly it's done so that it's easier to copy the command lines from the output of the execution trace and run them again.

Bash – Differences Between Reserved Word and Command time

In bash, time is a reserved word, so the shell can parse it the own way and apply rules for it.

Here is the code show how bash parse line start with time reserved word:

static int
time_command_acceptable ()
{
#if defined (COMMAND_TIMING)
  int i;

  if (posixly_correct && shell_compatibility_level > 41)
    {
      /* Quick check of the rest of the line to find the next token.  If it
     begins with a `-', Posix says to not return `time' as the token.
     This was interp 267. */
      i = shell_input_line_index;
      while (i < shell_input_line_len && (shell_input_line[i] == ' ' || shell_input_line[i] == '\t'))
        i++;
      if (shell_input_line[i] == '-')
    return 0;
    }

  switch (last_read_token)
    {
    case 0:
    case ';':
    case '\n':
    case AND_AND:
    case OR_OR:
    case '&':
    case WHILE:
    case DO:
    case UNTIL:
    case IF:
    case THEN:
    case ELIF:
    case ELSE:
    case '{':       /* } */
    case '(':       /* )( */
    case ')':       /* only valid in case statement */
    case BANG:      /* ! time pipeline */
    case TIME:      /* time time pipeline */
    case TIMEOPT:   /* time -p time pipeline */
    case TIMEIGN:   /* time -p -- ... */
      return 1;
    default:
      return 0;
    }
#else
  return 0;
#endif /* COMMAND_TIMING */
}

You see, time can be followed by most others bash reserved words.

In case of external command, the normal rule was applied, { was considered input of /usr/bin/time. } alone is invalid token, and bash raise the error.

In:

/usr/bin/time echo hello

external time did not call the shell builtin echo but the external echo command.

A strace verifies that:

$ strace -fe execve /usr/bin/time echo 1
execve("/usr/bin/time", ["/usr/bin/time", "echo", "1"], [/* 64 vars */]) = 0
Process 25161 attached
....
[pid 25161] execve("/usr/bin/echo", ["echo", "1"], [/* 64 vars */]) = -1 ENOENT (No such file or directory)
[pid 25161] execve("/bin/echo", ["echo", "1"], [/* 64 vars */]) = 0
1
[pid 25161] +++ exited with 0 +++
....

Here external time lookup your PATH variable to find the command executable. That also explain in case of using a function, you got No such file or directory becasue there's no command named mytest in your PATH.

Best Answer

Related Solutions

Bash – Why Does Bash Add Single Quotes to Unquoted Failed Pathname Expansions?

Bash – Differences Between Reserved Word and Command time

Related Question