Rsync re-copying all files

backupntfsrsync

I use ParagonNTFS on my Mac that I can write to ntfs drives.

I have two very large NTFS drives that I try to keep in sync. I'd like to be able to do this quickly with rsync:

rsync -rvh --delete /Volumes/Archive/ /Volumes/MobArchive/

Unfortunately, this command re-copies every file every time, including those that have not been updated. As a result, it takes all night to complete.

As a result, I have to use the –size-only option, which generally works as desired, except for files that have been updated but whose size has not changed.

I have read elsewhere that a common solution to this problem is to use –modify-window=1. In my case, however, this still causes all files to get copied. I have tried modify-window values all the way up to ten.

Any possible solutions?

Best Answer

If you change your command to start rsync -rvh ... to rsync -avh ... you should have more success.

The way you have the command just now rsync isn't copying over date stamps. The files in MobArchive will have their Create, Modify and Access times set to the time they are created in MobArchive. So every time rsync does a compare the date stamps in Archive and MobArchive will not match and it'll copy over everything again.

The "a" option (aka "archive" mode) is the equivalent of "-rlptgoD" which does a number of things including preserving timestamps.

Related Solutions

MacOS – Rsync with Linux server: special character problem

I finally figured out that apart from updating rsync, I also had to switch the order in which I specified the character sets: I thought you were supposed to specify the character set in the order of transformation; but it seems as that is not the correct syntax. Rather, one should always use --iconv=utf-8-mac,utf-8 when initialising the rsync from the mac, and always use --iconv=utf-8,utf-8-mac when initialising the rsync from the linux machine, no matter if I want to sync files from the mac or linux machine.

Then @Mark's solution works like magic!

MacOS – bug with rsync changing modification dates even with the -a option

Let me correct my comment: A 64bit timestamp consist of access-modification-change-birthtime.

From man 2 stat the following system calls change the respective times.

The time-related fields of struct stat are as follows:

 st_atime         Time when file data last accessed.  Changed by the mknod(2), utimes(2) and read(2) system calls.

 st_mtime         Time when file data last modified.  Changed by the mknod(2), utimes(2) and write(2) system calls.

 st_ctime         Time when file status was last changed (inode data modification).  Changed by the chmod(2), chown(2), link(2), mknod(2), rename(2),
                  unlink(2), utimes(2) and write(2) system calls.

 st_birthtime     Time of file creation. Only set once when the file is created. This field is only available in the 64 bit inode variants. On filesys-
                  tems where birthtime is not available, this field holds the ctime instead.

Tools such as cp,ditto, and pax can preserve OS X metadata when they are called to copy files. These tools will not preserve birthtime if the modification time is newer than the original file's birthtime. Birthtime of the new file is set to the modification time of the original file.

If you compile rsync with the patches fileflags, crtimes, hfs-compression then rsync can handle OS X metadata and preserve the original file's birthtime on the new file.

So, you would call rsync like this.

rsync -avXN --delete SOURCE DESTINATION

I suggest that you have a long read of the rsync manual and understand the options that I have suggested before you attempt to apply them.

Best Answer

Related Solutions

MacOS – Rsync with Linux server: special character problem

MacOS – bug with rsync changing modification dates even with the -a option

Related Question