Linux Environment Variable – How to Set Variable with Newline Characters

linuxstringvariable

I'm trying to set an RSA key as an environment variable which, as a text file, contains newline characters.

Whenever I attempt to read from the file and pass it into an environment variable, it will just stop at the newline character on the first line. How can I prevent this?

Best Answer

Note that except in zsh, shell variables cannot store arbitrary sequences of bytes. Variables in all other shells can't contain the NUL byte. And with the yash, they can't contain bytes not forming valid characters.

For files that don't contain NUL bytes, in POSIX-like shells, you can do:

var=$(cat file; echo .); var=${var%.}

We add a .\n and strip the trailing . to work around the fact that $(...) strips all trailing newline characters.

The above would also work in zsh for files that contain NULs though in zsh you could also use the $mapfile special associative array:

zmodload zsh/mapfile
var=$mapfile[file]

In zsh or bash, you can also use:

{ IFS= read -rd '' var || :; } < file

That reads up to the first NUL byte. It will return a non-zero exit status unless a NUL byte is found. We use the command group here to be able to at least tell the errors when opening the file, but we won't be able to detect read errors via the exit status.

Remember to quote that variable when passed to other commands. Newline is in the default value of $IFS, so would cause the variable content to be split when left unquoted in list contexts in POSIX-like shells other than zsh (not to mention the other problems with other characters of $IFS or wildcards).

So:

printf %s "$var"

for instance (not printf %s $var, certainly not echo $var which would add echo's problems in addition to the split+glob ones).

With non-POSIX shells:

Bourne shell:

The bourne shell did not support the $(...) form nor the ${var%pattern} operator, so it can be quite hard to achieve there. One approach is to use eval and quoting:

eval "var='`printf \\' | cat file - | awk -v RS=\\' -v ORS= -v b='\\\\' '
  NR > 1 {print RS b RS RS}; {print}; END {print RS}'`"

With (t)csh, it's even worse, see there.

With rc, you can use the ``(separator){...} form of command substitution with an empty separator list:

var = ``(){cat file}
Related Question