I don't know of any software that detects renames/moves across a whole folder hierarchy. That would be fairly slow, as it would have to scan all files to compare them.
The feature was once proposed as a patch for rsync (google for "rsync detect-renamed"), but apparently it was never accepted into rsync.
rsync does have a limited version of what you are looking for:
-y, --fuzzy
This option tells rsync that it should look for a basis file for any destination file that is missing. The current algorithm looks in the same directory as the destination file for either a file that has an identical size and modified-time, or a similarly-named file. If found, rsync uses the fuzzy basis file to try to speed up the transfer.
This will not work if you move files between directories, though.
That said, most sync programs do have the option to remove files in the destination if they are missing from the source, so a move should be replicated as new+delete, which requires more copying, but would work.
If you just want to compare drives, not sync them, you could just write a small script that lists all files with their filenames (and maybe a checksum), and import that into a spreadsheet or small db. Then you can sort by filename or checksum, and directly compare the lists of files. That might be the easiest route.
@Glytzhkof recommended Robocopy in his answer, and it suited my needs perfectly.
tl;dr
C:\>robocopy.exe source target /l /e /zb /xx /xl /fp /ns /nc /ndl /np /njh /njs
C:\source\foo\bar\b.txt
Details & Explanation of Options
Robocopy (Wikipedia) seems widely adopted for Windows system administration; is well-documented (TechNet); is discussed as more than an obscurity on Stack Overflow, Server Fault, and of course, here at Super User; provides for a specific function rather than trying to be a multi-purpose tool (which tend toward bloat and bugs); and furthermore has been providing this specific function since 1997. For me, all these factors contribute to "transparency," despite it being closed-source, and set my mind at ease.
Robocopy comes as part of a set of tools currently known as Windows Server 2003 Resource Kit Tools. After downloading and installing, I recreated the scenario in my question and gave it a go:
C:\>robocopy.exe source target /l /e /zb
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------
Started : Thu May 01 09:08:20 2014
Source : C:\source\
Dest : C:\target\
Files : *.*
Options : *.* /L /S /E /COPY:DAT /ZB /R:1000000 /W:30
------------------------------------------------------------------------------
0 C:\source\
1 C:\source\foo\
*EXTRA Dir -1 C:\target\foo\baz\
2 C:\source\foo\bar\
*EXTRA File 1 d.txt
Newer 5 b.txt
New File 1 c.txt
------------------------------------------------------------------------------
Total Copied Skipped Mismatch FAILED Extras
Dirs : 3 0 3 0 0 1
Files : 3 2 1 0 0 1
Bytes : 7 6 1 0 0 1
Times : 0:00:00 0:00:00 0:00:00 0:00:00
Ended : Thu May 01 09:08:20 2014
Looks good! Let me explain the options:
/l lists actions without actually carrying them out./e includes subdirectories, but unlike /s, includes empty directories too./zb copies in "restart" mode, and on access denied, "backup" mode; it seems like the safest approach; read more here.
I didn't need any of the copy-related options since I'm not actually performing any actions.
Anyway, next, it was only a matter of adding more switches to get the output I desired:
C:\>robocopy.exe source target /l /e /zb /xx /xl /fp /ns /nc /ndl /np /njh /njs
C:\source\foo\bar\b.txt
Again, let's go through the options.
First, I only cared about modified files and folders, so:
/xx excludes "extra" files and directories—those which exist only in the target./xl excludes "lonely" files and directories—those which exist only in the source.
Second, I desired relative paths (or at least full paths, not just names):
/fp enables full paths (unsurprisingly, there was no option for relative paths).
Third, I wanted to remove as much logging fluff as possible, and I was pleasantly surprised to find that all of it was removable:
/ns suppresses file sizes./nc suppresses classes, e.g. Newer./ndl suppresses directory names./np suppresses copy progress output./njh suppresses the job header./njs suppresses the job summary.
And there you have it!
For my purposes (creating versioned backups of changed files), I realized I'd actually like to have the timestamp of each modified file, too. Simply add /ts:
C:\>robocopy.exe source target /l /e /zb /xx /xl /fp /ns /nc /ndl /np /njh /njs /ts
2014/05/01 15:20:42 C:\source\foo\bar\b.txt
Best Answer
A script to move the files is below...
If you just want to see which files are the same/different, you can use
windiff. This might help with troubleshooting problems with the script.So, for your example:
Windiff will open and show which files are:
You can save the findings to a file using the command line option:
-S:Also, you can include
Xwith-Sto closewindiffafter writing the list like this:You can combine the lists, so if you want a list of the files that exist in only (any) one of the paths:
You can only generate one "log" file at a time, so if you wanted to generate all 4 individual "log" files you would have to run windiff 4 times:
Note: Files that exist in both paths, but located in different folders, will be shown as "leftonly" or "rightonly".
If you want a script to move the files that exist in only one of the paths to a different folder, you can use the batch script below.
Notes:
"domove=0") will only display "lonely" files without moving them. After you have tested the script and are confident that the correct files will be moved, you can change the value to: variable"domove=1"to have the "lonely" files displayed and moved.sdrive1,sdrive2,sfolder, andsdifffolderas necessary.spath1,spath2, andspathdiffif that is more appropriate for your use.I have made the following assumptions:
Here is the script:
Here is the output testing the script with the sample fileset you described: