Bash – Inconsistencies between redirecting input from file, here docs, and here strings

bashcommand-substitutionio-redirection

Why is this inconsistent? I would expect here docs and here strings to be functionally equivalent to redirecting input from a file.

$ bash --version
GNU bash, version 4.1.2(1)-release (x86_64-unknown-linux-gnu)
...

Expected output:

Prints each item from list.

$ for server in $(<servers.txt); do echo ${server}; done
server1
server2
server3
$

Unexpected output:

Prints nothing.

$ for server in $(<<EOF
> server1
> server2
> server3
> EOF
> ); do echo ${server}; done
$

Unexpected output:

Prints nothing.

$ for server in $(<<<"server1
> server2
> server3"); do echo ${server}; done
$

Edit: I traced bash while using an input redirector, and again with a here string. Here's the difference in behavior between the two:

Input redirector

$ echo $(<servers.txt)

...
31929 open("servers.txt", O_RDONLY)     = 3
31929 read(3, "server1\nserver2\nserver3\n", 128) = 24
31929 write(1, "server1\nserver2\nserver3\n", 24) = 24
...

This spawns child process 31929, which opens servers.txt as FD 3 and writes it to stdout.

Here string

$ echo $(<<<"server1
> server2
> server3")

...
31990 open("/tmp/sh-thd-106091305575", O_WRONLY|O_CREAT|O_EXCL|O_TRUNC, 0600) = 3
31990 write(3, "server1\nserver2\nserver3", 23) = 23
31990 write(3, "\n", 1)                 = 1
31990 open("/tmp/sh-thd-106091305575", O_RDONLY) = 4
31990 close(3)                          = 0
31990 unlink("/tmp/sh-thd-106091305575") = 0
31990 fcntl(0, F_GETFD)                 = 0
31990 fcntl(0, F_DUPFD, 10)             = 10
31990 fcntl(0, F_GETFD)                 = 0
31990 fcntl(10, F_SETFD, FD_CLOEXEC)    = 0
31990 dup2(4, 0)                        = 0
31990 close(4)                          = 0
31990 dup2(10, 0)                       = 0
31990 fcntl(10, F_GETFD)                = 0x1 (flags FD_CLOEXEC)
31990 close(10)
...

Several steps are performed here:

  1. Child process 31990 is spawned
  2. The here string is written to /tmp/sh-thd-106091305575
  3. /tmp/sh-thd-106091305575 is opened as read-only (FD 4)
  4. /tmp/sh-thd-106091305575 is unlinked (so when FD 4 is closed it will be deleted)
  5. stdin is duped to FD 10
  6. FD 4 (the temp file containing our here string) is duped to FD 0 (stdin) This is where a process — in this case the bash subshell — would act on stdin
  7. FD 4 is closed and FD 10 is duped back to FD 0

The surprising thing is that this ever happens in the input redirector example:

31929 write(1, "server1\nserver2\nserver3\n", 24) = 24

I suspect that bash has special behavior for this case. Will probably need to dig into the source for a definitive answer.

Best Answer

$(<servers.txt) is a special shortcut in bash for $(cat <servers.txt), as documented under “Command substitution”. It is an exception to the regular syntax where the redirection would be applied to an empty command and thus do nothing except signal an error if the file doesn't exist. Bash doesn't extend this exception to here documents or here strings: you need to include cat explicitly.

This feature was added in bash 2.02 (it's mentioned in the changelog). In the source code, it's implemented by the logic around the call to cat_file in parse_and_execute in builtins/evalstring.c.

Related Question