Shell – How to wait for the copying process (cp) to finish

cpprocessraspberry pishell-script

I want my RPi 2 (with MiniBian (a minimal version of Debian-based Raspbian OS) as its OS) to execute a script when it detects a USB flash memory is inserted. So this is what I have done:

I created /etc/udev/rules.d/10-usbstick.rules with these contents:

ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd*" SYMLINK+="usbflash", RUN+="/path/to/myscript.sh"

And then created /path/to/myscript.sh with these contents:

#!/bin/bash

mount /dev/usbflash /mnt/

service omxd stop

rm -r /path/to/myfolder/*
cp -f /mnt/*.mp4 /path/to/myfolder/
cp -f /mnt/*.avi /path/to/myfolder/

sleep 1
umount /dev/usbflash

halt

Now, when I insert a USB flash memory, it recognizes it, mounts it, stops the omxd daemon and removes all the older files.

But the problems is that it shuts down (halt) before all the files have been copied. I check later and see that only 1 file has been copied, and it has not been copied correctly, like the halt command has been executed at the middle of the copying procedure.

So, I decided to use wait command to ensure that the halt command be executed only at the end. I edited the /path/to/myscript.sh file as this:

#!/bin/bash

mount /dev/usbflash /mnt/

PID1 = $!
wait PID1

service omxd stop

rm -r /path/to/myfolder/*

PID2 = $!
wait PID2

cp -f /mnt/*.mp4 /path/to/myfolder/

PID3 = $!
wait PID3

cp -f /mnt/*.avi /path/to/myfolder/

PID4 = $!
wait PID4

sleep 1
umount /dev/usbflash

PID5 = $!
wait PID5

halt

But again the same scenario happens: the system halts at the middle of the copying procedure.

So what's wrong with my system?

What shall I do to tell the system to halt only after all the new files have been successfully copied !?

Update 1

Adding sync before umount and halt didn't work. The same scenario happens again.

Update 2

I edited the /path/to/myscript.sh file as the following:

mkdir -p /mnt/usb
if mount /dev/usbflash /mnt/usb
then
    service omxd stop
    rm -r /path/to/myfolder/*
    cp -f /mnt/usb/*.mp4 /path/to/myfolder/
    cp -f /mnt/usb/*.avi /path/to/myfolder/
    sync
    umount /mnt/usb
    sync
    shutdown -h now
fi

But it didn't help either! The same scenario happens again! And this time it was worse: it didn't copy any file at all!

Update 3

I changed the /etc/udev/rules.d/10-usbstick.rules to the following:

ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd[a-z]1" SYMLINK+="usbflash", RUN+="/path/to/myscript.sh"

Now it works great. It copies all the files. But there's a new problem: It doesn't shut down!

Update 4

I found something new:

When I run myscript.sh directly from the terminal, it works perfect: removes the older files, copies the new files, and shuts down the system. Perfect.

So, why doesn't it execute myscript.sh perfectly when I call it from udev rules?

And is there any other way to shut down the system after it has copied the files !?

Best Answer

I don't know why changing the kernel name would have made a difference (perhaps sd*1 runs after sd* allowing a bit more time for work to get done?), but udev doesn't like long-running actions in events:

Starting daemons or other long running processes is not appropriate for udev; the forked processes, detached or not, will be unconditionally killed after the event handling has finished.

Original `nohup` suggestion

I originally suggested nohup before fully reading my own links :) -- which suggest that this may not actually work

udev rule:

ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd[a-z]1" SYMLINK+="usbflash", RUN+="/path/to/mywrapper.sh"

mywrapper.sh (Note: if you don't redirect the output nohup may litter your directory with a nohup.out file):

#!/bin/sh
nohup /path/to/myscript.sh >/log/myscript.log 2>&1 &

Then myscript.sh can be what it is.

Newer `systemd` suggestion

The third link below suggests firing off a systemd service when the device is plugged in. That blog entry does more work than I think you need due to ensuring that the device information is passed to the service, I think you can get away with simply using systemd as your method of daemonization:

ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd[a-z]1" SYMLINK+="usbflash", RUN+="/usr/bin/systemctl start my-usb-backup.service"

With a simple oneshot service my-usb-backup.service:

[Unit]
Description=run myscript
[Service]
Type=oneshot
ExecStart=/path/to/myscript.sh

Example

A modified version of your example code, cmd.bash.

!/bin/bash

array[0]=1
array[0]=2

sleep 30 &
pid=$!

pidArr=()
for i in ${array[@]}; do
  for n in $(seq 3); do
    if [ $n -eq 1 ]; then
      sleep 31 > file &
      pidArr+=($!)
    else
      sleep 32 >> file &
      pidArr+=($!)
    fi
  done
done

echo ${pidArr[@]}

I've substituted sleep commands in this example just to simulate some long running jobs that we can background. They have nothing to do with this other than to act as stand-ins.

Also the final echo command is there purely so we can see what happens when the for loops complete. Ultimately we'll replace it with an actual wait command, but let's not get ahead of ourselves.

Example run #1

So let's run our program and see what happens.

$ ./cmd.bash 
24666 24667 24668

Now if we check ps:

$ ps -eaf|grep sleep
saml     24664     1  0 22:28 pts/9    00:00:00 sleep 30
saml     24666     1  0 22:28 pts/9    00:00:00 sleep 31
saml     24667     1  0 22:28 pts/9    00:00:00 sleep 32
saml     24668     1  0 22:28 pts/9    00:00:00 sleep 32

If we wait a bit, these will ultimately finish. But this shows that if we add the process IDs to an array we can echo them out afterwards.

Example #2

So let's change that echo line out and swap in a wait line now, something like this:

  wait ${pidArr[@]}

Now when we run our modified version we see that it's waiting on ALL the process IDs:

$ ./cmd.bash

...after ~30 seconds passes

We get back our prompt. So we successfully waited for all the processes. A quick check of ps:

$ ps -eaf|grep sleep
$

So it worked.

Best Answer

Original nohup suggestion

Newer systemd suggestion

Related Solutions

Shell – Wait for process to finish before going to the next line in shell script

Bash – Wait for background processes to finish with inverse match process ID

Example

Example run #1

Example #2

Related Question

Original `nohup` suggestion

Newer `systemd` suggestion