Bash – Reading a binary file as an array of bytes or 16 or 32 bits using shell script

bashlinuxshellshell-script

I am working on ARM-based modem Linux host machine and I need to read a bin file as an 8-,16- or 32- bits array. I basically need to encrypt the file in question and was thinking of just XOR-ing the bits with a random bit sequence (I don't need a very complex system). The Linux host doesn't have C or PERL modules in it so I cannot use them and hence I need to do it using a shell script. Can anyone give me some pointers on how to proceed with this?

Best Answer

Use tr (here rot128):

LC_ALL=C tr '\0-\377' '\200-\377\0-\177' < infile > outfile

For XOR 123, you'd need to compute the corresponding tr line:

LC_ALL=C tr '\0-\377' '\173\172\171\170\177\176\175\174\163\162\161\160\167\166\165\164\153\152\151\150\157\156\155\154\143\142\141\140\147\146\145\144\133\132\131\130\137\136\135\134\123\122\121\120\127\126\125\124\113\112\111\110\117\116\115\114\103\102\101\100\107\106\105\104\73\72\71\70\77\76\75\74\63\62\61\60\67\66\65\64\53\52\51\50\57\56\55\54\43\42\41\40\47\46\45\44\33\32\31\30\37\36\35\34\23\22\21\20\27\26\25\24\13\12\11\10\17\16\15\14\3\2\1\0\7\6\5\4\373\372\371\370\377\376\375\374\363\362\361\360\367\366\365\364\353\352\351\350\357\356\355\354\343\342\341\340\347\346\345\344\333\332\331\330\337\336\335\334\323\322\321\320\327\326\325\324\313\312\311\310\317\316\315\314\303\302\301\300\307\306\305\304\273\272\271\270\277\276\275\274\263\262\261\260\267\266\265\264\253\252\251\250\257\256\255\254\243\242\241\240\247\246\245\244\233\232\231\230\237\236\235\234\223\222\221\220\227\226\225\224\213\212\211\210\217\216\215\214\203\202\201\200\207\206\205\204'

More generally, to answer the question, to convert a file into an array of numbers to be used by the shell with standard commands:

set -- $(od -An -vtu1 < infile)

You can then apply the transformations you want and convert back to a file with awk's printf("%c").

That's going to be very inefficient though.

Like:

for i in $(od -An -vtu1 < infile); do
  echo "$(($i ^ 123))"
done | awk '{printf "%c", $0}' > outfile

You can use od -An -vtu2, to get 16bit numbers, but note that it is in the local endianness, so when converting back to characters, you have to take that into account.

Related Solutions

Bash – Passing named arguments as array in shell script

If on Linux (with the util-linux utilities including getopt installed, or the one from busybox), you can do:

declare -A opt_spec
var1=() var2=() var4=false
unset var3
opt_spec=(
  [opt1:]='var1()' # opt with argument, array variable
  [opt2:]='var2()' # ditto
  [opt3:]='var3'   # opt with argument, scalar variable
  [opt4]='var4'    # boolean opt without argument
)
parsed_opts=$(
  IFS=,
  getopt -o + -l "${!opt_spec[*]}" -- "$@"
) || exit
eval "set -- $parsed_opts"
while [ "$#" -gt 0 ]; do
  o=$1; shift
  case $o in
    (--) break;;
    (--*)
      o=${o#--}
      if ((${opt_spec[$o]+1})); then # opt without argument
        eval "${opt_spec[$o]}=true"
      else
        o=$o:
        case "${opt_spec[$o]}" in
          (*'()') eval "${opt_spec[$o]%??}+=(\"\$1\")";;
          (*) eval "${opt_spec[$o]}=\$1"
        esac
        shift
      fi
  esac
done
echo "var1: ${var1[@]}"

That way, you can call your script as:

my-script --opt1=foo --opt2 bar --opt4 -- whatever

And getopt will do the hard work of parsing it, handling -- and abbreviations for you.

Alternatively, you could rely on the type of the variable instead of specifying it in your $opt_spec associative array definition:

declare -A opt_spec
var1=() var2=() var4=false
unset var3
opt_spec=(
  [opt1:]=var1   # opt with argument
  [opt2:]=var2   # ditto
  [opt3:]=var3   # ditto
  [opt4]=var4    # boolean opt without argument
)
parsed_opts=$(
  IFS=,
  getopt -o + -l "${!opt_spec[*]}" -- "$@"
) || exit
eval "set -- $parsed_opts"
while [ "$#" -gt 0 ]; do
  o=$1; shift
  case $o in
    (--) break;;
    (--*)
      o=${o#--}
      if ((${opt_spec[$o]+1})); then # opt without argument
        eval "${opt_spec[$o]}=true"
      else
        o=$o:
        case $(declare -p "${opt_spec[$o]}" 2> /dev/null) in
          ("declare -a"*) eval "${opt_spec[$o]}+=(\"\$1\")";;
          (*) eval "${opt_spec[$o]}=\$1"
        esac
        shift
      fi
  esac
done
echo "var1: ${var1[@]}"

You can add short options like:

declare -A long_opt_spec short_opt_spec
var1=() var2=() var4=false
unset var3
long_opt_spec=(
  [opt1:]=var1   # opt with argument
  [opt2:]=var2   # ditto
  [opt3:]=var3   # ditto
  [opt4]=var4    # boolean opt without argument
)
short_opt_spec=(
  [a:]=var1
  [b:]=var2
  [c]=var3
  [d]=var4
)
parsed_opts=$(
  IFS=; short_opts="${!short_opt_spec[*]}"
  IFS=,
  getopt -o "+$short_opts" -l "${!long_opt_spec[*]}" -- "$@"
) || exit
eval "set -- $parsed_opts"
while [ "$#" -gt 0 ]; do
  o=$1; shift
  case $o in
    (--) break;;
    (--*)
      o=${o#--}
      if ((${long_opt_spec[$o]+1})); then # opt without argument
        eval "${long_opt_spec[$o]}=true"
      else
        o=$o:
        case $(declare -p "${long_opt_spec[$o]}" 2> /dev/null) in
          ("declare -a"*) eval "${long_opt_spec[$o]}+=(\"\$1\")";;
          (*) eval "${long_opt_spec[$o]}=\$1"
        esac
        shift
      fi;;
    (-*)
      o=${o#-}
      if ((${short_opt_spec[$o]+1})); then # opt without argument
        eval "${short_opt_spec[$o]}=true"
      else
        o=$o:
        case $(declare -p "${short_opt_spec[$o]}" 2> /dev/null) in
          ("declare -a"*) eval "${short_opt_spec[$o]}+=(\"\$1\")";;
          (*) eval "${short_opt_spec[$o]}=\$1"
        esac
        shift
      fi
  esac
done
echo "var1: ${var1[@]}"

Bash – Accessing Array Index Variable in Shell Script Loop

You can do this using List of array keys. From the bash man page:

${!name[@]}
${!name[*]}

List of array keys. If name is an array variable, expands to the list of array indices (keys) assigned in name. If name is not an array, expands to 0 if name is set and null otherwise. When @ is used and the expansion appears within double quotes, each key expands to a separate word.

For your example:

#!/bin/bash
AR=('foo' 'bar' 'baz' 'bat')
for i in "${!AR[@]}"; do
  printf '${AR[%s]}=%s\n' "$i" "${AR[i]}"
done

This results in:

${AR[0]}=foo
${AR[1]}=bar
${AR[2]}=baz
${AR[3]}=bat

Note that this also work for non-successive indexes:

#!/bin/bash
AR=([3]='foo' [5]='bar' [25]='baz' [7]='bat')
for i in "${!AR[@]}"; do
  printf '${AR[%s]}=%s\n' "$i" "${AR[i]}"
done

This results in:

${AR[3]}=foo
${AR[5]}=bar
${AR[7]}=bat
${AR[25]}=baz

Best Answer

Related Solutions

Bash – Passing named arguments as array in shell script

Bash – Accessing Array Index Variable in Shell Script Loop

Related Question