How to disable word splitting during command substitution

command-substitutionzsh

How can I disable word-splitting during command substitution? Here's a simplified example of the problem:

 4:00PM /Users/paymahn/Downloads
 ❯❯❯ cat test.txt
hello\nworld
 4:00PM /Users/paymahn/Downloads
 ❯❯❯ echo $(cat test.txt )
hello
world
 4:00PM /Users/paymahn/Downloads
 ❯❯❯ echo "$(cat test.txt )"
hello
world
 4:01PM /Users/paymahn/Downloads
 ❯❯❯ echo "$(cat "test.txt" )"
hello
world

What I want is for echo $(cat test.txt) (or some variant of that which includes command subsitution) to output hello\nworld.

I found https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html which says at the bottom If the substitution appears within double quotes, word splitting and filename expansion are not performed on the results. but I can't seem to make sense of that. I would have thought that one of the examples I already tried conformed to that rule but I guess not.

Best Answer

Having a literal \n get changed to a newline isn't about word splitting, but echo processing the backslash. Some versions of echo do that, some don't... Bash's echo doesn't process backslash-escapes by default (without the -e flag or xpg_echo option), but e.g. dash's and Zsh's versions of echo do.

$ cat test.txt 
hello\nworld
$ bash -c 'echo "$(cat test.txt)"'
hello\nworld
$ zsh -c 'echo "$(cat test.txt)"'
hello
world

Use printf instead:

$ bash -c 'printf "%s\n" "$(cat test.txt)"'
hello\nworld
$ zsh -c 'printf "%s\n" "$(cat test.txt)"'
hello\nworld

Regardless of that, you should put the quotes around the command substitution to prevent word splitting and globbing in sh-like shells. (zsh only does word splitting (not globbing) upon command substitution (not upon parameter or arithmetic expansions) except in sh-mode.)

Related Solutions

Bash – Command Substitution Splitting on Newline but Not Space

Portably:

set -f              # turn off globbing
IFS='
'                   # split at newlines only
cmd $(cat <file)
unset IFS
set +f

Or using a subshell to make the IFS and option changes local:

( set -f; IFS='
'; exec cmd $(cat <file) )

The shell performs field splitting and filename generation on the result of a variable or command substitution that is not in double quotes. So you need to turn off filename generation with set -f, and configure field splitting with IFS to make only newlines separate fields.

There's not much to be gained with bash or ksh constructs. You can make IFS local to a function, but not set -f.

In bash or ksh93, you can store the fields in an array, if you need to pass them to multiple commands. You need to control expansion at the time you build the array. Then "${a[@]}" expands to the elements of the array, one per word.

set -f; IFS=$'\n'
a=($(cat <file))
set +f; unset IFS
cmd "${a[@]}"

Bash – Word-splitting when parameter is used within command substitution

You need to put double quotes around all variable and command substitutions.

set -- "$(basename -- "$file")"

If you allow the value of the variable to be split into words and these words to be treated as glob patterns, there's no going back. You can no longer tell how much whitespace was there or whether the file names were the result of a wildcard expansion. You can't turn the hamburger back into a cow, so you must make sure not to send the cow to the abattoir in the first place.

Best Answer

Related Solutions

Bash – Command Substitution Splitting on Newline but Not Space

Bash – Word-splitting when parameter is used within command substitution

Related Question