Ubuntu – Search for duplicate photos

deleteduplicate filesperlsearch

I have photos, in folder A, they also exist, in folders B and C. They have the same filenames. How to search for them and delete them from folder A? A perl script or something would be ideal since I don't like to download a software for this task.

Best Answer

Try this command:

find . -type f -exec cmp -s '{}' "$destdir/{}" \; -exec echo mv -v "$destdir/{}" "$ToDelete"/ \;

How to use?

Step 1: Make a temporary directory to moving duplicate files into that:

ToDelete=/tmp/ToDelete; mkdir -p "$ToDelete"

Step 2: Set destdir to your FolderA directory which we want to delete duplicate photos from there.

destdir=/path/to/FolderA

Step 3 cd to your first source directory(FolderB) and run the command

cd /path/to/FolderB

find . -type f -exec cmp -s '{}' "$destdir/{}" \; -exec echo mv -v "$destdir/{}" "$ToDelete"/ \;

Step 4: Now cd to your next source directory(FolderC) and run again the command

cd /path/to/FolderC

find . -type f -exec cmp -s '{}' "$destdir/{}" \; -exec echo mv -v "$destdir/{}" "$ToDelete"/ \;

Ok. Now if you test the command and see the result, so remove the echo command to moving the duplicate photos into /tmp/ToDelete directory.

See the test below:

$ ls ~/FolderA    
  1.jpg  2.jpg  6.jpg  7.jpg  8.jpg

$ ls ~/FolderB    
  3.jpg  4.jpg  5.jpg  7.jpg  8.jpg  9.jpg

$ ls ~/FolderC
  6.jpg  7.jpg  8.jpg  9.jpg

$ ToDelete=/tmp/ToDelete; mkdir -p "$ToDelete"
$ ls /tmp/ToDelete/
$ destdir=~/FolderA

$ cd ~/FolderB
/FolderB$ find . -type f -exec cmp -s '{}' "$destdir/{}" \; -exec mv -v "$destdir/{}" "$ToDelete"/ \;
  ‘/home/Fischer/FolderA/./8.jpg’ -> ‘/tmp/ToDelete/8.jpg’
  removed ‘/home/Fischer/FolderA/./8.jpg’
  ‘/home/Fischer/FolderA/./7.jpg’ -> ‘/tmp/ToDelete/7.jpg’
  removed ‘/home/Fischer/FolderA/./7.jpg’

/FolderB$ cd ~/FolderC
/FolderC$ find . -type f -exec cmp -s '{}' "$destdir/{}" \; -exec mv -v "$destdir/{}" "$ToDelete"/ \;
  ‘/home/Fischer/FolderA/./6.jpg’ -> ‘/tmp/ToDelete/6.jpg’
  removed ‘/home/Fischer/FolderA/./6.jpg’

/FolderC$ ls /tmp/ToDelete/
  6.jpg  7.jpg  8.jpg

/FolderC$ ls -l ~/FolderA
  1.jpg  2.jpg

/FolderC$ C00L ;)

And How it works?

When you switch into your FolderB or FolderC and you run find command, so you are looking for any file in it (. refer to current directory) and then with cmp command(used to compare two files byte by byte) you compare each files({}) in FolderB/FolderC with files in destdir(FolderA($destdir/{})) if they are the same then we move that file(from FolderA to temp directory in /tmp/ToDelete) with next mv command -exec mv "$destdir/{}" "$ToDelete"/ \;.

Related Question