Bash – How to run multiple scripts in parallel

bashparallelismperlshell-script

I have one bash script which calls the same perl script in a serial way. The bash script is used to collect the overall results, while the perl script collects the results of my simulations for the given attributes.

The bash script looks as follows: 

mkdir ./results/csv     && \
../perlscripts/v2csv.pl -v -F reach results/Heterogeneous*.vec > ./results/csv/reach.csv
../perlscripts/v2csv.pl -v -F roundTrip results/Heterogeneous*.vec > ./results/csv/RT.csv
../perlscripts/v2csv.pl -v -F downlink results/Heterogeneous*.vec > ./results/csv/DL.csv
../perlscripts/v2csv.pl -v -F clusters results/Heterogeneous*.vec > ./results/csv/clusters.csv

Collecting the results by calling one perl script at a time is really long; I am looking for a way which would allow me to call the different variations of the perl script within the bash script in parallel. Is there a way to achieve this in bash?

Just to clarify, I don't want the commands which call the perl script to be dependent on each other in any way. I want all of them to start at the same point in time, as if I had 4 separate bash-terminals each executing one of these commands.

Similar:
https://stackoverflow.com/questions/15644991/running-several-scripts-in-parallel-bash-script

Best Answer

If you have gnu parallel installed, you could make a script with just the commands, e.g.:

../perlscripts/v2csv.pl -v -F reach results/Heterogeneous*.vec > ./results/csv/reach.csv
../perlscripts/v2csv.pl -v -F roundTrip results/Heterogeneous*.vec > ./results/csv/RT.csv
../perlscripts/v2csv.pl -v -F downlink results/Heterogeneous*.vec > ./results/csv/DL.csv
../perlscripts/v2csv.pl -v -F clusters results/Heterogeneous*.vec > ./results/csv/clusters.csv

and then run them in parallel:

mkdir ./results/csv && parallel :::: myscript.sh

Alternatively, invoking the command and using {} - the default replacement string:

mkdir ./results/csv && parallel ../perlscripts/v2csv.pl -v -F {} \
results/Heterogeneous*.vec '>' ./results/csv/{}.csv ::: reach roundTrip downlink clusters

would run the following commands in parallel:

../perlscripts/v2csv.pl -v -F reach results/Heterogeneous*.vec > ./results/csv/reach.csv
../perlscripts/v2csv.pl -v -F roundTrip results/Heterogeneous*.vec > ./results/csv/roundTrip.csv
../perlscripts/v2csv.pl -v -F downlink results/Heterogeneous*.vec > ./results/csv/downlink.csv
../perlscripts/v2csv.pl -v -F clusters results/Heterogeneous*.vec > ./results/csv/clusters.csv
Related Question