How to compare the ls output of two folders to find a missing directory

files

I'm trying to compare a folder with 1400 subfolders to one with a 1.399. I need to know which subfolder is missing.

I tried this:

diff -rq dir1/ dir2/ | grep dir1/ | awk '{print $4}' > difference1.txt

But it's been like 6 hours yet and no output. ls in the folders is nearly instant so is there a faster approach than diff?

Best Answer

diff -u <(ls dir1) <(ls dir2)

This will make sure diff does not look inside the subdirs.

Or try this if you are brave ;)

diff -u <(find dir1/ -maxdepth 1 -type d -exec basename {} \;) <(find dir2/ -maxdepth 1 -type d -exec basename {} \;)

Related Solutions

Rsync – How to Compare Two Folders and Copy Differences

I am not sure whether you can do it with any existing linux commands such as rsync or diff. But in my case I had to write my own script using Python, as python has the "filecmp" module for file comparison. I have posted the whole script and usage in my personal site - http://linuxfreelancer.com/

It usage is simple - give it the absolute path of new directory, old directory and difference directory in that order.

#!/usr/bin/env python

import os, sys
import filecmp
import re
from distutils import dir_util
import shutil

holderlist = []


def compareme(dir1, dir2):
    dircomp = filecmp.dircmp(dir1, dir2)
    only_in_one = dircomp.left_only
    diff_in_one = dircomp.diff_files
    dirpath = os.path.abspath(dir1)
    [holderlist.append(os.path.abspath(os.path.join(dir1, x))) for x in only_in_one]
    [holderlist.append(os.path.abspath(os.path.join(dir1, x))) for x in diff_in_one]
    if len(dircomp.common_dirs) > 0:
        for item in dircomp.common_dirs:
            compareme(
                os.path.abspath(os.path.join(dir1, item)),
                os.path.abspath(os.path.join(dir2, item)),
            )
        return holderlist


def main():
    if len(sys.argv) > 3:
        dir1 = sys.argv[1]
        dir2 = sys.argv[2]
        dir3 = sys.argv[3]
    else:
        print "Usage: ", sys.argv[0], "currentdir olddir difference"
        sys.exit(1)

    if not dir3.endswith("/"):
        dir3 = dir3 + "/"

    source_files = compareme(dir1, dir2)
    dir1 = os.path.abspath(dir1)
    dir3 = os.path.abspath(dir3)
    destination_files = []
    new_dirs_create = []
    for item in source_files:
        destination_files.append(re.sub(dir1, dir3, item))
    for item in destination_files:
        new_dirs_create.append(os.path.split(item)[0])
    for mydir in set(new_dirs_create):
        if not os.path.exists(mydir):
            os.makedirs(mydir)
    # copy pair
    copy_pair = zip(source_files, destination_files)
    for item in copy_pair:
        if os.path.isfile(item[0]):
            shutil.copyfile(item[0], item[1])


if __name__ == "__main__":
    main()

How to compare two directories thoroughly without git

diff -Naur dir1/ dir2/

Pretty useful for submitting software patches, incidentally: diff -Naur orig/ patched/ > my_changes.diff

Best Answer

Related Solutions

Rsync – How to Compare Two Folders and Copy Differences

How to compare two directories thoroughly without git

Related Question