Shell Scripting – How to Run Commands in Parallel and Wait for Completion

background-processprocess-managementshellshell-scriptwait

I have script including multiple commands. How can I group commands to run together ( I want to make several groups of commands. Within each group, the commands should run in parallel (at the same time). The groups should run sequentially, waiting for one group to finish before starting the next group)
… i.e.

#!/bin/bash
command #1
command #2
command #3
command #4
command #5
command #6
command #7
command #8
command #9
command #10

how can I run every 3 commands to gether? I tried:

#!/bin/bash
{
command #1
command #2
command #3
} & 
{   
command #4
command #5
command #6
} & 
{
command #7
command #8
command #9
}&
command #10

But this didn't work properly ( I want to run the groups of commands in parallel at the same time. Also I need to wait for the first group to finish before running the next group)

The script is exiting with an error message!

Best Answer

The commands within each group run in parallel, and the groups run sequentially, each group of parallel commands waiting for the previous group to finish before starting execution.

The following is a working example:

Assume 3 groups of commands as in the code below. In each group the three commands are started in the background with &.

The 3 commands will be started almost at the same time and run in parallel while the script waits for them to finish.

After all three commands in the the third group exit, command 10 will execute. 

$ cat command_groups.sh 
#!/bin/sh

command() {
    echo $1 start
    sleep $(( $1 & 03 ))      # keep the seconds value within 0-3
    echo $1 complete
}

echo First Group:
command 1 &
command 2 &
command 3 &
wait

echo Second Group:
command 4 &
command 5 &
command 6 &
wait

echo Third Group:
command 7 &
command 8 &
command 9 &
wait

echo Not really a group, no need for background/wait:
command 10

$ sh command_groups.sh 
First Group:
1 start
2 start
3 start
1 complete
2 complete
3 complete
Second Group:
4 start
5 start
6 start
4 complete
5 complete
6 complete
Third Group:
7 start
8 start
9 start
8 complete
9 complete
7 complete
Not really a group, no need for background/wait:
10 start
10 complete
$
Related Question