Bash Scripting – NUL Delimited Variable

bashcommand-substitution

GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)

Idea is to set a variable to a NUL delimited data set. Here $samples

This, however, result in:

warning: command substitution: ignored null byte in input

when doing:

samples="$(find . -type d -iregex './sample[0-9][0-9]' -printf "%f\0" | sort -z)"

Thought I could re-use this variable as I need to iterate the same values multiple times:

while IFS= read -rd '' sample; do
    echo $sample
done<<< "$samples"

I could use \n over \0 in the find command in this exact case, but would like to know how, if possible, to do it with NUL delimiter generally speaking.

Optionally I could do:

while IFS= read -rd '' sample; do
    echo $sample
done< <(find . -type d -iregex './E[0-9][0-9]' -printf "%f\0" | sort -z)

but – as I need to loop it several times it makes for some very redundant code – and would have to run the find and sort command each time.

Convert the result into an array perhaps?

  • Is this possible?
  • Why can not NUL delimited data be used as is?

Best Answer

It is a fact that you can't store \0 null bytes in a bash string context, because of the underlying C implementation. See Why $'\0' or $'\x0' is an empty string? Should be the null-character, isn't it?.

One option would be strip off the null bytes after the sort command, at the end of the pipeline using tr and store the result to solve the immediate problem of the warning message thrown. But that would still leave your logic flawed as the filenames with newlines would still be broken.

Use an array, use the mapfile or readarray command (on bash 4.4+) to directly slurp in the results from the find command

IFS= readarray -t -d '' samples < <(find . -type d -iregex './sample[0-9][0-9]' -printf "%f\0" | sort -z)
Related Question