Edge cases in filesystem operations during backup and file copy

backupconcurrencyfilesystemsrsync

I have a question about file access in Linux that I cannot sum it up completely.

Consider that I have a home directory that I backup it using rsync triggered via cron. My home directory is on an EXT4 file system and I'm logged in during the rsync run.

My question is what happens if a file is modified during mid-backup (while rsync is reading it). AFAIK EXT fs family doesn't have any measures against it and that will corrupt (or render it meaningless) the backup of the file I just took.

Is my theory correct or am I missing a small locking mechanism which guarantees sound backups.

Best Answer

Using plain rsync files are read the same way any application would read a file. This can lead to copies with inconsistent data. The best way to prevent inconsistent copies is using LVM snapshots, which will prevent changes to the data while copying.

From my experience, getting inconsistent data is rare, likely due to the way the the kernel buffers writes. Only very heavily written files, such as database files, result in corruption.

Related Solutions

How to include and exclude with rsync to back up the system

So, the --exclude-from file, will be:

var/tmp/
var/cache/

and use rsync like this:

rsync -aAv --delete --stats --exclude-from /excludes /home /var /usr/local /etc /Backup/

Rsync Special Files – Understanding Rsync’s Relationship with Special Files

It is not --special that "should" sync the devices, it is the --devices indirect switch. For that the man page says:

--devices
       This option causes rsync to transfer character and block  device
       files  to  the  remote  system  to recreate these devices.  This
       option has no effect if the receiving rsync is not  run  as  the
       super-user (see also the --super and --fake-super options).

and you don't seem to be logging in as root on the remote system, which makes the option have no effect.

Best Answer

Related Solutions

How to include and exclude with rsync to back up the system

Rsync Special Files – Understanding Rsync’s Relationship with Special Files

Related Question