Ubuntu – Find duplicate files and folders and move these to a different folder

filesystemguisoftware-recommendation

Possible Duplicate:
How to find (and delete) duplicate files

Is there a reliable duplicate file/folders utility, (with GUI) for Linux that can find duplicate files or folders and move them to different folder?

Best Answer

fdupes

No GUI but fdupes / sudo apt-get install fdupes is very fast and reliable. It uses sizes and modification dates for a preliminary analysis, then compares md5 hashes of the files and then does a bit compare if necessary. It's also dead easy to use. I strongly recommend it.

Typical usage:

fdupes -d -r /path/to/directory/

-r for walking subdirectories as opposed to walking just the contents of the specified dir.

-d to prompt the user about which file to delete (without this fdupes just compiles a list of duplicated)

-N deletes without prompt

-H normally, when two or more files point to the same disk area they are treated as non-duplicates; this option will change this behavior

-L hardlink duplicate files to the first file in each set of duplicates without prompting the user (this option was rolled back in some versions as it was found to be buggy and unsafe in rare cases. It might be reintroduced in future versions).

Edit: The hardlink options was removed as buggy for now. It might return some day. For now you have to use hardlink / sudo apt-get install hardlink

fslint

If you insist on a graphical user interface you might want to have a look at fslint / sudo apt-get install fslint (see website for description). It is more feature rich but also more complicated and less reliable.

Related Solutions

Ubuntu – How to find (and delete) duplicate files

I use fdupes for this. It is a commandline program which can be installed from the repositories with sudo apt install fdupes. You can call it like fdupes -r /dir/ect/ory and it will print out a list of dupes. fdupes has also a simple Homepage and a Wikipedia article, which lists some more programs.

Ubuntu – How to count the total number of files/folders on a system

Since file / folder names can contain newlines:

sudo find / -type f -printf '.' | wc -c
sudo find / -type d -printf '.' | wc -c

This will count any file / folder in the current / directory. But as muru points out you might want to exclude virtual / other filesystems from the count (the following will exclude any other mounted filesystem):

find / -xdev -type f -printf '.' | wc -c
find / -xdev -type d -printf '.' | wc -c

sudo find / -type f -printf '.': prints a dot for each file in /;
sudo find / -type d -printf '.': prints a dot for each folder in /;
wc -c: counts the number of characters.

Here's an example of how not taking care of newlines in file / folder names may break other methods such as e.g. find / -type f | wc -l and how using find / -type f -printf '.' | wc -c actually makes it right:

% ls
% touch "file
\`dquote> with newline"
% find . -type f | wc -l
2
% find . -type f -printf '.' | wc -c
1

If STDOUT is not a terminal, find will print each file / folder name literally; this means that a file / folder name containing a newline will be printed across two different lines, and that wc -l will count two lines for a single file / folder, ultimately printing a result off by one.