Shell – How to run multiple pv commands in parallel

job-controlparallelismpvshell

I want to run a sequence of command pipelines with pv on each one. Here's an example:

for p in 1 2 3
do
  cat /dev/zero | pv -N $p | dd of=/dev/null &
done

The actual commands in the pipe don't matter (cat/dd are just an example)…

The goal being 4 concurrently running pipelines, each with their own pv output. However when I try to background the commands like this, pv stops and all I get are 4 stopped jobs. I've tried with {...|pv|...}&, bash -c "...|pv|..." & all with the same result.

How can I run multiple pv command pipelines concurrently?

Best Answer

Found that I can do this with xargs and the -P option:

josh@subdivisions:/# seq 1 10 | xargs -P 4 -I {} bash -c "dd if=/dev/zero bs=1024 count=10000000 | pv -c -N {} | dd of=/dev/null"
        3: 7.35GiB 0:00:29 [ 280MiB/s] [                                                                                         <=>                                                                 ]
        1: 7.88GiB 0:00:29 [ 312MiB/s] [                                                                                         <=>                                                                 ]
        4: 7.83GiB 0:00:29 [ 258MiB/s] [                                                                                         <=>                                                                 ]
        2: 6.55GiB 0:00:29 [ 238MiB/s] [                                                                                         <=>                                                                 ]

Send output of the array to iterate over into stdin of xargs; To run all commands simultaneously, use -P 0

Related Question