Prevent backup from running in parallel

backuprsync

In my script /usr/local/bin/backup, that I call every hour from /etc/crontab, I use rsync to copy data to an off-site server.
That all worked fine, even in cases where we had somewhat more new data than can be pushed out in an hour.

Last week someone copied an 11GB file on the data partition and when I found out the next day there were 14 rsync programs running in parallel, each of then getting no bandwidth and each probably working on the same huge file. I killed them all (before realising I should have kept the first one running), stopped the cron job and ran the backup script by hand.

I can write out a file in the script before starting rsync and check in the script if that file is already there to prevent backup from running in parallel. Is there an easier way of doing this?

My /etc/crontab entry:

5  *  *  *  *   root  /usr/local/bin/backup

Best Answer

There are different ways of doing this, but IMO the easiest is inserting flock before the command in the crontab file:

5  *  *  *  *   root  flock -n /var/lock/backup /usr/local/bin/backup

The /var/lock/backup file is the lock that flock uses and -n immediately makes the command fail if the lock already exists.

This could of course mean that if one backup takes 1 hour and 1 minute, that the next one starts 59 minutes later. If that is a problem you could look into using -x.

Related Question