Ubuntu – Find and execute multiple scripts in parallel

bashcommand linefindscripts

G'day,

I have tried to solve this problem all morning but have not had success. I would like to search for certain scripts in a particular folder and execute them all in parallel. The scripts have different names, but all end with "run.sh". Below what I have tried without success.

The first approach was to use find in combination with -execdir. This works, however, the scripts are executed in a sequantial fashion. I want all scripts to be executed simultaneously (parallel). There seems to be no option to achieve this with -execdir

find . -name "*run.sh" -type f -execdir 'nohup' {} '&' \;

Then, I tried using xargs because there is has a parallel option (-P). I have not tried the parallel option yet, because I am unable to get xargs to run the scripts in their respective subfolders. The commands below execute all scripts, but do that in the folder I am running the command in, therefore the scripts themselves do not work. The scripts have to be executed in their own subfolders. In the above example, -execdir is doing that as opposed to -exec. How do I achieve that with xargs?

nohup find . -name "*run.sh" -type f | xargs -0 -I{} bash -c f\ \{\}

nohup find . -name "*run.sh" -type f | xargs -0 -I{} bash -c "f {}"

nohup find . -name "*run.sh" -type f | xargs -0 -I{} bash -c "./{}"

I am really frustrated and am hoping that there is somebody out there that can help.
Thank you so much!

Best Answer

Inspired by choroba's answer:

find . -iname '*run.sh' -printf 'cd %h; ./%f\0' | xargs -L1 -0 -P0 bash -c

You can use -printf to build the command line. %h is the directory where the file is located, and %f is the name of the file without the path (the basename). The -P option enables parallelism in xargs, and -L1 -0 makes it use one null-terminated line of input per command.

Bash aliases

Aliases are the shortcuts in the command line. The regular shell, Bash, allows you to create your own aliases.

To try it out (without saving them permanently), try this:

alias myalias='gksudo gparted'
myalias

To save them permanently, create a file ~/.bash_aliases like this:

alias myalias='gksudo gparted'
alias edsources='gksudo gedit /etc/apt/sources.list'

The default ~/.bashrc file installed for users in Ubuntu will source the ~/.bash_aliases file by default.

Aliases allow you to append more arguments and 'overrule' current commands too. A common use case of this is

alias rm='rm -i'

To get a confirmation message for everything you try to delete with rm.

In case you want to 'escape' an alias by running the original command, prepend it with a backslash. So, with the rm example, just do

\rm myfile

to remove the file without the -i option forced by the alias.

Current script

About your current approach, you should just be able to prepend the command in the one-line script with sudo (or gksudo).

Ubuntu – Filenames with spaces breaking for loop, find command

Using for with find is the wrong approach here, see for example this writeup about the can of worms you are opening.

The recommended approach is to use find, while and read as described here. Below is an example that should work for you:

find . -type f -name '*.*' -print0 | 
while IFS= read -r -d '' file; do
    printf '%s\n' "$file"
done

This way you delimit the filenames with null (\0) characters, this means that variation in space and other special characters will not cause problems.

In order to update an archive with the files that find locates, you can pass its output directly to tar:

find . -type f -name '*.*' -printf '%p\0' | 
tar --null -uf archive.tar -T -

Note that you do not have to differentiate between if the archive exists or not, tar will handle it sensibly. Also note the use of -printf here to avoid including the ./ bit in the archive.

Best Answer

Related Solutions

Ubuntu – How to execute scripts as root without pointing to its location

Bash aliases

Current script

Ubuntu – Filenames with spaces breaking for loop, find command

Related Question