Shell – How to capture stdin to a variable without stripping any trailing newlines

echonewlinesprintfshellshell-script

In a shell script…

How do I capture stdin to a variable without stripping any trailing newlines?

Right now I have tried:

var=`cat`
var=`tee`
var=$(tee)

In all cases $var will not have the trailing newline of the input stream. Thanks.

ALSO: If there is no trailing newline in the input, then the solution must not add one.

UPDATE IN LIGHT OF THE ACCEPTED ANSWER:

The final solution that I used in my code is as follows:

function filter() {
    #do lots of sed operations
    #see https://github.com/gistya/expandr for full code
}

GIT_INPUT=`cat; echo x`
FILTERED_OUTPUT=$(printf '%s' "$GIT_INPUT" | filter)
FILTERED_OUTPUT=${FILTERED_OUTPUT%x}
printf '%s' "$FILTERED_OUTPUT"

If you would like to see the full code, please see the github page for expandr, a little open-source git keyword-expansion filter shell script that I developed for information security purposes. According to rules set up in .gitattributes files (which can be branch-specific) and git config, git pipes each file through the expandr.sh shell script whenever checking it in or out of the repository. (That is why it was critical to preserve any trailing newlines, or lack thereof.) This lets you cleanse sensitive information, and swap in different sets of environment-specific values for test, staging, and live branches.

Best Answer

The trailing newlines are stripped before the value is stored in the variable. You may want to do something like:

var=`cat; echo x`

and use ${var%x} instead of $var. For instance:

printf "%s" "${var%x}"

Note that this solves the trailing newlines issue, but not the null byte one (if standard input is not text), since according to POSIX command substitution:

If the output contains any null bytes, the behavior is unspecified.

But shell implementations may preserve null bytes.

Related Question