Bash Scripting – Sending Null Byte in Unix Pipe

bashio-redirectionnull

I am trying to redirect python generated input to ELF 64-bit executable in bash 5.0.3. I am getting:

> ./bf <<< $(python2 -c "print('c'*6+b'\x00'+'c'*6)")
bash: warning: command substitution: ignored null byte in input
Enter password: Password didn't match
input: cccccccccccc

How can I allow a null byte in the input?

Best Answer

You are able to pass null bytes across a pipe (like you say in the title), but the bash shell will not allow null bytes in expansions. It does not allow null bytes in expansions because the shell uses C strings to represent the results of expansions, and C strings are terminated by null bytes.

$ hexdump -C <<< $( python2 -c "print('c'*6+b'\x00'+'c'*6)" )
bash: warning: command substitution: ignored null byte in input
00000000  63 63 63 63 63 63 63 63  63 63 63 63 0a           |cccccccccccc.|
0000000d

Passing the data across a pipe is fine:

$ python2 -c "print('c'*6+b'\x00'+'c'*6)" | hexdump -C
00000000  63 63 63 63 63 63 00 63  63 63 63 63 63 0a        |cccccc.cccccc.|
0000000e

Redirecting a process substitution also works, as process substitutions don't expand to the data produced by the command but to the name of a file containing that data:

$ hexdump -C < <( python2 -c "print('c'*6+b'\x00'+'c'*6)" )
00000000  63 63 63 63 63 63 00 63  63 63 63 63 63 0a        |cccccc.cccccc.|
0000000e

So, the solution is to avoid having the shell store the data containing the null byte in a string, and instead pass the data over a pipe, without using a command substitution. In your case

$ python2 -c "print('c'*6+b'\x00'+'c'*6)" | ./bf

Or switch to zsh which does allow null bytes in strings:

$ hexdump -C <<< $( python2 -c "print('c'*6+b'\x00'+'c'*6)" )
00000000  63 63 63 63 63 63 00 63  63 63 63 63 63 0a        |cccccc.cccccc.|
0000000e

Related Solutions

Bash – In bash, how to convert 8 bytes to an unsigned int (64bit LE)

Bash is the wrong tool altogether. Shells are good at gluing bits and pieces together; text processing and arithmetic are provided on the side, and data processing isn't in their purview at all.

I'd go for Python over Perl, because Python has bignums right off the bat. Use struct.unpack to unpack the data.

#!/usr/bin/env python
import os, struct, sys
fmt = "<" + "Q" * 8192
header_bytes = sys.stdin.read(65536)
header_ints = list(struct.unpack(fmt, header_bytes))
sys.stdin.seek(-65536, 2)
footer_bytes = sys.stdin.read(65536)
footer_ints = list(struct.unpack(fmt, header_bytes))
# your calculations here

Here's my answer to the original question. The revised question doesn't have much to do with the original, which was about converting one 8-byte sequence into the 64-bit integer it represents in little-endian order.

I don't think bash has any built-in feature for this. The following snippet sets a to a string that is the hexadecimal representation of the number that corresponds to the bytes in the specified string in big endian order.

a=0x$(printf "%s" "$string" |
      od -t x1 -An |
      tr -dc '[:alnum:]')

For little-endian order, reverse the order of the bytes in the original string. In bash, and for a string of known length, you can do

a=0x$(printf "%s" "${string:7:1}${string:6:1}${string:5:1}${string:4:1}${string:3:1}${string:2:1}${string:1:1}${string:0:1}" |
      od -t x1 -An |
      tr -dc '[:alnum:]')

You can also get your platform's prefered endianness if your od supports 8-byte types.

a=0x$(printf "%s" "$string" |
      od -t x8 -An |
      tr -dc '[:alnum:]')

Whether you can do arithmetic on $a will depend on whether your bash supports 8-byte arithmetic. Even if it does, it'll treat it as a signed value.

Alternatively, use Perl:

a=0x$(perl -e 'print unpack "Q<", $ARGV[0]' "$string")

If your perl is compiled without 64-bit integer support, you'll need to break the bytes up.

a=0x$(perl -e 'printf "%x%08x\n", reverse unpack "L<L<", $ARGV[0]' "$string")

(Replace < by > for big-endian or remove it to get the platform endianness.)

Shell Script – Redirecting to /dev/null

No, this will not prevent the script from crashing. If any errors occur in the tar process (e.g.: permission denied, no such file or directory, ...) the script will still crash.

This is because of using > /dev/null 2>&1 will redirect all your command output (both stdout and stderr) to /dev/null, meaning no outputs are printed to the terminal.

By default:

stdin  ==> fd 0
stdout ==> fd 1
stderr ==> fd 2

In the script, you use > /dev/null causing:

stdin  ==> fd 0
stdout ==> /dev/null
stderr ==> fd 2

And then 2>&1 causing:

stdin  ==> fd 0
stdout ==> /dev/null
stderr ==> stdout

Best Answer

Related Solutions

Bash – In bash, how to convert 8 bytes to an unsigned int (64bit LE)

Shell Script – Redirecting to /dev/null

Related Question