Robocopy copying unchanged files to other server

backuprobocopy

I have a problem with a robocopy script that is supposed to mirror a directory including the NTFS ACL from one server to another server.

It seems that even though no files have changed, the complete content is copied every time.

This is the command I am using

robocopy \\abc.com\original \\otherserver\mirrordir /XD _MirrorLogs /MIR /COPY:DATS /R:2 /W:2 /LOG+:\abc.com\logfile.LOG /TEE

This is the result (extracts) I am getting

-------------------------------------------------------------------------------
ROBOCOPY     ::     Robust File Copy for Windows                              
-------------------------------------------------------------------------------

Started : Wed Jun 12 23:00:19 2013

Source : \\abc.com\original\
Dest : \\otherserver\mirrordir\

Files : *.*

Exc Dirs : _MirrorLogs

Options : *.* /TEE /S /E /DCOPY:DA /COPY:DATS /PURGE /MIR /R:2 /W:2 

------------------------------------------------------------------------------

...

                   2    \\abc.com\original\Best_Practices\
            7223    file1.html
          194048    file2.doc
                   3    \\abc.com\original\_history\
          155194    file3.xlsm
    New File              159091    file4.xlsm
  0%  
 20%  
 41%  
 61%  
 82%  
100%  
    New File          155222    file5.xlsm
  0%  
 21%  
 42%  
 63%  
 84%  
100%  

...

------------------------------------------------------------------------------

           Total    Copied   Skipped  Mismatch    FAILED    Extras
 Dirs :       422         0       422         0         0         0
Files :      3123      3123         0         0         0         0
Bytes :  649.27 m  649.27 m         0         0         0         0
Times :   0:56:17   0:52:18                       0:00:00   0:03:58


Speed :              216918 Bytes/sec.
Speed :              12.412 MegaBytes/min.

Ended : Wed Jun 12 23:56:38 2013

I can understand that when there is the tag "New File" that this needs to be copied. Okay.
What I also understand is that no Dirs are copied according to the summary. This is what I expected.
What I don't understand is that it seems that all files were copied again, even though there were no changes. Most of the files look like file1.html, file2.doc and file3.xlsm, i.e. there is no information why the file is copied (like New File or Newer).
I checked with Beyond Compare if the files are identical, and from the content point of view they are. I thought about different access rights, but then why does the problem not occur in case of the directories (where the access rights were also copied).

Does anyone have a hint for me?

Thanks in advance!

Best Answer

The short answer is that when robocopy is copying these folders from one server to another, I suspect that it is seeing the timestamps differently than you expect, so it is deciding that the existing files on the destination are older than the same files on the source. This could be due to differences in the time/timezone/DST settings on the two machines.

When robocopy analyzes the files before copying, it classifies each file into one of these classes: 

Lonely files:  exist on source but not destination.
Newer files:   have Newer timestamp on source, (size and attributes: N/A).
Older files:   have Older timestamp on source, (size and attributes: N/A).
Changed files: have same timestamp, but different size (attributes: N/A).
Same files:    have same timestamp, size, and attributes.
Tweaked files: have same timestamp and size, but different attributes.
Extra files:   exist on destination but not source.
Mismatched:    is a file on one, and a directory on the other, in source
               and destination.

By default (and with the command line options you have given), robocopy will copy Lonely, Newer, and Changed files. Files in the other classes will not be copied, and will either be simply reported in the output and/or log, or completly ignored.

If robocopy is copying a file and overwriting an existing file on the destination, then robocopy thinks that either the timestamo of the source file is before (Newer) the timestamp of the destination file, or the file sizes are different (Changed).

In order to tell exactly what's happening to each file, you could run robocopy with the "/L" and "/V" command line options. Using the "/L" options tells robocopy to not do any actual copying, and only list what would otherwise be copied. Using the "/V" options tells robocopy to to provide Verbose details.

This will show why robocopy has decided to copy (and skip) each file. So, for example using robocopy to copy an example fileset.

Here is a listing of the files:

 Directory of C:\folder1

02/28/2014  12:48 PM               327 File1.txt
02/28/2014  02:03 PM               333 File2.txt
               2 File(s)            660 bytes

 Directory of C:\folder2

02/28/2014  12:43 PM               327 File1.txt
02/28/2014  02:03 PM               327 File2.txt
               2 File(s)            654 bytes

In this example, the files named File1.txt, are the same size in the source and destination folders, but has a Newer timestamp in Folder1. And, the files named File2.txt, have the same timestamp in the source and destination folders, but have different (Changed) file sizes.

Running robocopy to copy Folder1 to Folder2, with the "/L" and "/V" command line options looks like this.

C:\>robocopy "C:folder1" "c:folder2" /E /L /V

-------------------------------------------------------------------------------
   ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------

  Started : Fri Feb 28 21:07:29 2014

   Source : C:\folder1\
     Dest : C:\folder2\

    Files : *.*

  Options : *.* /V /L /S /E /COPY:DAT /R:1000000 /W:30

------------------------------------------------------------------------------

                           2    C:\folder1\
            Newer                    327        File1.txt
            Changed                  333        File2.txt

------------------------------------------------------------------------------

               Total    Copied   Skipped  Mismatch    FAILED    Extras
    Dirs :         1         0         1         0         0         0
   Files :         2         2         0         0         0         0
   Bytes :       660       660         0         0         0         0
   Times :   0:00:00   0:00:00                       0:00:00   0:00:00

   Ended : Fri Feb 28 21:07:29 2014

If you run robocopy with the "/L" and "/V" command line options on your set of files, it should give you the information of why robocopy is unexpectedly copying these files.

I suspect that robocopy is seeing the timestamps differently than you expect, and is deciding that the files on the source are Newer than the same existing files on the destination. This could be due to differences in the time/timezone/DST settings on the two machines.

Related Question