How to Compose Bash Functions Using Pipes

bashpipeshell-script

I have few functions defined in this fashion:

function f {
  read and process $1
  ...
  echo $result
}

I want to compose them together so that invocation would look like f | g | h.

What idiom shoud I use to convert function working on arguments to one reading arguments from stdin? Is it possible to read pairs, tuples of arguments from stream without need to escape them (e.g. null terminating)?

Best Answer

One potential approach would be to put a while...read construct inside your functions which would process any data that came into the function through STDIN, operate on it, and then emit the resulting data back out via STDOUT.

function X {
  while read data; do
    ...process...
  done
}

Care will need to be spent with how you configure your while ..read.. components since they'll be highly dependent on the types of data they'll be able to reliably consume. There may be an optimal configuration that you can come up with.

Example

$ logF() { while read data; do echo "[F:$(date +"%D %T")] $data"; done; }
$ logG() { while read data; do echo "G:$data";                    done; }
$ logH() { while read data; do echo "H:$data";                    done; }

Here's each function by itself.

$ echo "hi" | logF
[F:02/07/14 20:01:11] hi

$ echo "hi" | logG
G:hi

$ echo "hi" | logH
H:hi

Here they are when we use them together.

$ echo "hi" | logF | logG | logH
H:G:[F:02/07/14 19:58:18] hi

$ echo -e "hi\nbye" | logF | logG | logH
H:G:[F:02/07/14 19:58:22] hi
H:G:[F:02/07/14 19:58:22] bye

They can take various styles of input.

#-- ex. #1
$ cat <<<"some string of nonsense" | logF | logG | logH
H:G:[F:02/07/14 20:03:47] some string of nonsense

#-- ex. #2    
$ (logF | logG | logH) <<<"Here comes another string."
H:G:[F:02/07/14 20:04:46] Here comes another string.

#-- ex. #3
$ (logF | logG | logH)
Look I can even
H:G:[F:02/07/14 20:05:19] Look I can even
type to it
H:G:[F:02/07/14 20:05:23] type to it
live
H:G:[F:02/07/14 20:05:25] live
via STDIN
H:G:[F:02/07/14 20:05:29] via STDIN
..type Ctrl + D to stop..

#-- ex. #4
$ seq 5 | logF | logG | logH
H:G:[F:02/07/14 20:07:40] 1
H:G:[F:02/07/14 20:07:40] 2
H:G:[F:02/07/14 20:07:40] 3
H:G:[F:02/07/14 20:07:40] 4
H:G:[F:02/07/14 20:07:40] 5

#-- ex. #5
$ (logF | logG | logH) < <(seq 2)
H:G:[F:02/07/14 20:15:17] 1
H:G:[F:02/07/14 20:15:17] 2