Bash – ENV variables in relation to xargs loops

bashenvironment-variablesxargs

Are env variables (set using export) treated differently when the same script is run multiple times simultaneously? That's a weird question, so here's an example I built out which is similar to my actual problem.

hello.sh

#!/bin/bash
export COUNTER=$((COUNTER+1));
echo $COUNTER;
sleep 5;
/hello.sh

nums.txt

1
2
3
4
5

bash

# cat /nums.txt | xargs -L 1 -P 5 /hello.sh

expected output

1
2
3
4
5
   (...5 seconds later)
6
7
8
9
10

actual output

1
1
1
1
1
   (... 5 seconds later)
2
2
2
2
2

So running it just once, works fine. But when I start to run it multiple time in parallel is when i witness this behavior. I assume it's for a reason but I have no clue how to prevent it.

To add background, my goal here is so that when each script finishes, it starts back up with the next increment read from $COUNTER. This way I can set it once starting with 5 separate threads, and each of those will continually increment and call the script.

Best Answer

Yes, you are right, this is similar to "How can I make environment variables "exported" in a shell script stick around?".

If you define a variable as:

COUNTER=$((COUNTER+1))

then it exists in the current shell only. It will not be seen either by subshells this shell creates or by the calling shell. When using export:

export COUNTER=$((COUNTER+1))

then the variable is also seen by this shell's subshells.

When you create 5 processes with xargs, they each inherit the environment of the calling shell. They however do not share any changes to the environment with each other.

Related Question