Linux – How to diff directories for different files, but not line-by-line

Use the comp command under cmd.exe

[C:\]comp /?
Compares the contents of two files or sets of files.                                                                    

COMP [data1] [data2] [/D] [/A] [/L] [/N=number] [/C] [/OFF[LINE]]                                                       

  data1      Specifies location and name(s) of first file(s) to compare.                                                
  data2      Specifies location and name(s) of second files to compare.                                                 
  /D         Displays differences in decimal format.                                                                    
  /A         Displays differences in ASCII characters.                                                                  
  /L         Displays line numbers for differences.                                                                     
  /N=number  Compares only the first specified number of lines in each file.                                            
  /C         Disregards case of ASCII letters when comparing files.                                                     
  /OFF[LINE] Do not skip files with offline attribute set.                                                              

To compare sets of files, use wildcards in data1 and data2 parameters. 

cmp does things byte-by-byte, so it probably won't run out of memory (just tested it on two 7 GB files) -- but you might be looking for more detail than a list of "files X and Y differ at byte x, line y". If the similarities of your files are offset (e.g., file Y has an identical block of text, but not at the same location), you can pass offsets to cmp; you could probably turn it into a resynchronizing compare with a small script.

Aside: In case anyone else lands here when looking for a way to confirm that two directory structures (containing very large files) are identical: diff --recursive --brief (or diff -r -q for short, or maybe even diff -rq) will work and not run out of memory.