Bash – Echo vs <<<, or Useless Use of Echo Award?

bashio-redirectionperformancepipe

By now the Useless Use of cat Award is very well known, and there's also a mention of a Useless Use of echo (not relevant for this question). I'm wondering if there should be a "Useless Use of echo in Bash Award": Piping seems to be much slower than heredocs and herestrings according to some highly unscientific measurements:

  • Heredocs:

    for reps in 1 2 3
do
    time for i in {1..1000}
    do
        cat <<'END'
test string
END
    done > /dev/null
done

real    0m1.786s
user    0m0.212s
sys     0m0.332s

real    0m1.817s
user    0m0.232s
sys     0m0.332s

real    0m1.846s
user    0m0.256s
sys     0m0.320s

  • Herestrings

    for reps in 1 2 3
do
    time for i in {1..1000}
    do
        cat <<< 'test string'
    done > /dev/null
done

real    0m1.932s
user    0m0.280s
sys     0m0.288s

real    0m1.956s
user    0m0.248s
sys     0m0.348s

real    0m1.968s
user    0m0.268s
sys     0m0.324s

  • Redirection

    for reps in 1 2 3
do
    time for i in {1..1000}
    do
        echo 'test string' | cat
    done > /dev/null
done

real    0m3.562s
user    0m0.416s
sys     0m0.548s

real    0m3.924s
user    0m0.384s
sys     0m0.604s

real    0m3.343s
user    0m0.400s
sys     0m0.552s

In general, heredocs and herestrings are about the same speed (this is just one data set from several tests) while redirection is consistently more than 50% slower. Am I misunderstanding something, or could this be used as a general rule for commands reading standard input in Bash?

Best Answer

First, let's concentrate on performance. I ran benchmarks for a slightly different program on an otherwise mostly idle x86_64 processor running Debian squeeze.

herestring.bash, using a herestring to pass a line of input:

#! /bin/bash
i=0
while [ $i -lt $1 ]; do
  tr a-z A-Z <<<'hello world'
  i=$((i+1))
done >/dev/null

heredoc.bash, using a heredoc to pass a line of input:

#! /bin/bash
i=0
while [ $i -lt $1 ]; do
  tr a-z A-Z <<'EOF'
hello world
EOF
  i=$((i+1))
done >/dev/null

echo.bash, using echo and a pipe to pass a line of input:

#! /bin/bash
i=0
while [ $i -lt $1 ]; do
  echo 'hello world' | tr a-z A-Z
  i=$((i+1))
done >/dev/null

For comparison, I also timed the scripts under ATT ksh93 and under dash (except for herestring.bash, because dash doesn't have herestrings).

Here are median-of-three times:

$ time bash ./herestring.bash 10000
./herestring.bash 10000  0.32s user 0.79s system 15% cpu 7.088 total
$ time ksh ./herestring.bash 10000
ksh ./herestring.bash 10000  0.54s user 0.41s system 17% cpu 5.277 total
$ time bash ./heredoc.bash 10000
./heredoc.bash 10000  0.35s user 0.75s system 17% cpu 6.406 total
$ time ksh ./heredoc.bash 10000  
ksh ./heredoc.sh 10000  0.54s user 0.44s system 19% cpu 4.925 total
$ time sh ./heredoc.bash 10000  
./heredoc.sh 10000  0.08s user 0.58s system 12% cpu 5.313 total
$ time bash ./echo.bash 10000
./echo.bash 10000  0.36s user 1.40s system 20% cpu 8.641 total
$ time ksh ./echo.bash 10000
ksh ./echo.sh 10000  0.47s user 1.51s system 28% cpu 6.918 total
$ time sh ./echo.sh 10000
./echo.sh 10000  0.07s user 1.00s system 16% cpu 6.463 total

Conclusions:

  • A heredoc is faster than a herestring.
  • echo and a pipe is noticeably, but not dramatically faster. (Keep in mind that this is a toy program: in a real program, most of the processing time would be in whatever the tr call stands for here.)
  • If you want speed, ditch bash and call dash or even better ksh instead. Bash's features don't make up for its relative slowness, but ksh has both features and speed.

Beyond performance, there's also clarity and portability. <<< is a ksh93/bash/zsh extension which is less well-known than echo … | or <<. It doesn't work in ksh88/pdksh or in POSIX sh.

The only place where <<< is arguably significantly clearer is inside a heredoc:

foo=$(tr a-z A-Z <<<'hello world')

vs

foo=$(tr a-z A-Z <<'EOF'
hello world
EOF
)

(Most shells can't cope with closing the parenthesis at the end of the line containing <<EOF.)

Related Question