Ubuntu – sudo rsync -a does not preserve mtime

rsyncUbuntu

In order to debug my repeatedly crashing Ubuntu 16.04 I want to backup /var/log from my HDD to an external USB medium. I am running an live OS (Ubuntu 16.04) and using the –archive option: 

$ sudo rsync --archive /media/ubuntu/2f1ea741-cc93-4b0e-ad74-1e21d56d27a9/var/log/* /media/ubuntu/transcend/DebuggingSystemCrash@box3/var_log_2016-07-04-0939

Expected behavior

Archive mode equals -rlptgoD, so modification times should be preserved (which I want), since 

-t, --times                 preserve modification times

Observed behavior

The time stamps of the files (atime and mtime) are not being preserved, see screenshot. mtime of folders is preserved.

Screenshot with source on the left and destination on the right

Since both source and target are mounted this question is probably related to how to tell rsync to preserve time stamp on files when source tree has a mounted point.

I have also read:

Am I missing something?

UPDATE

Not running rsync as sudo produces the expected behavior regarding mtime, but will skip some files.

comparison of rsync output: $ sudo rsync -a vs $ rsync -a

Best Answer

Difference with sudo could be due to the extract below from Documentation/filesystems/vfat.txt. (Requires knowledge that your target filesystem is of type vfat & who owns the directory^Wfilesystem - determined by mount option in the case of vfat).

I went ahead and posted this anyway because it shows how important the filesystem types are. The general conclusion was that using FAT on Linux is pain (and there has been a special issue with rsync on FAT as well).

I'm certain at least one of the issues listed there has been solved with different defaults (possibly in udisks as used by the GUI, as opposed to manual mount commands). In any case I suggest you'd be wasting your time if what you're trying to do is copy parts of your Linux OS into FAT, and you demand specific results in file metadata (timestamps), or even file names. In that case you should use a native Linux filesystem, ideally the same type as the source.

Or, instead of copying individual files, create an archive using a native Linux tool such as tar. Think Zip file - but specifically designed for backing up Linux *nix files - their names, and the most common metadata. For example, tar -c -f out.tar input-directory/ (omits compression).

Reference doc for setting FAT timestamps on Linux

allow_utime=### -- This option controls the permission check of mtime/atime.

                  20 - If current process is in group of file's group ID,
                       you can change timestamp.
                   2 - Other users can change timestamp.

                 The default is set from `dmask' option. (If the directory is
                 writable, utime(2) is also allowed. I.e. ~dmask & 022)

                 Normally utime(2) checks current process is owner of
                 the file, or it has CAP_FOWNER capability.  But FAT
                 filesystem doesn't have uid/gid on disk, so normal
                 check is too unflexible. With this option you can
                 relax it.

Community confirmation and other potential issues to avoid

So the blog post I found confirms "only the owner of a FAT32 mount can set file time stamps on it". I'd call this a design bug (root should be able to do it as well), which the reference documentation almost spells out, but I'm too lazy to submit a patch for it.

The blog also provides an explanation for a file/directory difference. Unfortunately it seems to be doubly opposite what you describe. I think the file/directory difference described below should have prevented your normal user from setting timestamps correctly on directories - whereas what you've showed is that if you're root, you can set the correct timestamps on directories but not files.

It sounds to me like the --modify-window=1 is a workaround for an rsync-specific issue. So using cp to copy files instead might give you another data point.

Even the owner of a FAT32 mount can't set directory times reliably

Although I was now able to copy over the original file time stamps correctly I noticed that all of the directory modification times were set to the time I ran the rsync command, which is not what I wanted at all. A little googling uncovered this forum posting about this problem: http://ubuntuforums.org/showthread.php?t=886048

This forum posting suggested adding the option "--modify-window=1", which gives 1 second slack on how closely file and directory times have to match before rsync will see them as different, and someone said that worked to correctly preserve original directory timestamps.

Related Question