Shell Script – Bring HEREDOC Text into a Variable

posixshell-script

I'm trying to bring HEREDOC text into a shell script variable in a POSIX compliant way. I tried like so:

#!/bin/sh

NEWLINE="
"

read_heredoc2() {
  while IFS="$NEWLINE" read -r read_heredoc_line; do
    echo "${read_heredoc_line}"
  done
}

read_heredoc2_result="$(read_heredoc2 <<'HEREDOC'

                        _                            _ _
                       | |                          | (_)
  _ __ ___  _   _ _ __ | | __ _  ___ ___  ___  _ __ | |_ _ __   ___
 | '_ ` _ \| | | | '_ \| |/ _` |/ __/ _ \/ _ \| '_ \| | | '_ \ / _ \
 | | | | | | |_| | |_) | | (_| | (_|  __/ (_) | | | | | | | | |  __/
 |_| |_| |_|\__, | .__/|_|\__,_|\___\___|\___/|_| |_|_|_|_| |_|\___|
             __/ | |
            |___/|_|



HEREDOC
)"

echo "${read_heredoc2_result}"

That produced the following which is wrong:

                        _                            _ _
                       | |                          | (_)
  _ __ ___  _   _ _ __ | | __ _  ___ ___  ___  _ __ | |_ _ __   ___
 | '_ ` _ \| | | | '_ \| |/ _` |/ __/ _ \/ _ \| '_ \| | | '_ \ / _  | | | | | | |_| | |_) | | (_| | (_|  __/ (_) | | | | | | | | |  __/
 |_| |_| |_|\__, | .__/|_|\__,_|\___\___|\___/|_| |_|_|_|_| |_|\___|
             __/ | |
            |___/|_|

The following works but I don't like how clunky it is by using a random output variable:

#!/bin/sh

NEWLINE="
"

read_heredoc1() {
  read_heredoc_first=1
  read_heredoc_result=""
  while IFS="$NEWLINE" read -r read_heredoc_line; do
    if [ ${read_heredoc_first} -eq 1 ]; then
      read_heredoc_result="${read_heredoc_line}"
      read_heredoc_first=0
    else
      read_heredoc_result="${read_heredoc_result}${NEWLINE}${read_heredoc_line}"
    fi
  done
}

read_heredoc1 <<'HEREDOC'

                        _                            _ _            
                       | |                          | (_)           
  _ __ ___  _   _ _ __ | | __ _  ___ ___  ___  _ __ | |_ _ __   ___ 
 | '_ ` _ \| | | | '_ \| |/ _` |/ __/ _ \/ _ \| '_ \| | | '_ \ / _ \
 | | | | | | |_| | |_) | | (_| | (_|  __/ (_) | | | | | | | | |  __/
 |_| |_| |_|\__, | .__/|_|\__,_|\___\___|\___/|_| |_|_|_|_| |_|\___|
             __/ | |                                                
            |___/|_|                                                



HEREDOC

echo "${read_heredoc_result}"

Correct output:

                        _                            _ _            
                       | |                          | (_)           
  _ __ ___  _   _ _ __ | | __ _  ___ ___  ___  _ __ | |_ _ __   ___ 
 | '_ ` _ \| | | | '_ \| |/ _` |/ __/ _ \/ _ \| '_ \| | | '_ \ / _ \
 | | | | | | |_| | |_) | | (_| | (_|  __/ (_) | | | | | | | | |  __/
 |_| |_| |_|\__, | .__/|_|\__,_|\___\___|\___/|_| |_|_|_|_| |_|\___|
             __/ | |                                                
            |___/|_|

Any ideas?

Best Answer

The problem is that, in Bash, inside $( ... ) escape (and other) sequences get parsed, even though the heredoc itself wouldn't have them. You get a doubled line because \ escapes the line break. What you're seeing is really a parsing issue in Bash - other shells don't do this. Backticks can also be a problem in older versions. I have confirmed that this is a bug in Bash, and it will be fixed in future versions.

You can at least simplify your function drastically:

func() {
    res=$(cat)
}
func <<'HEREDOC'
...
HEREDOC

If you want to choose the output variable it can be parameterised:

func() {
    eval "$1"'=$(cat)'
}
func res<<'HEREDOC'
...
HEREDOC

Or a fairly ugly one without eval:

{ res=$(cat) ; } <<'HEREDOC'
...
HEREDOC

The {} are needed, rather than (), so that the variable remains available afterwards.

Depending on how often you'll do this, and to what end, you might prefer one or another of these options. The last one is the most concise for a one-off.

If you're able to use zsh, your original command substitution + heredoc will work as-is, but you can also collapse all of this down further:

x=$(<<'EOT'
...
EOT
)

Bash doesn't support this and I don't think any other shell that would experience the problem you're having does either.

Related Question