Comparing two folders contents with automator (quick action)

automator

I work in a video company. Very often, we copy a heavy load of files from SD cards to our own folders, and before we format the SD cards, we want to check for contents on both directories if they are the same.

I've done some "Quick actions" earlier but this one is too tricky for me. What I want to do is compare the SD directory and the media folder (each one has different structure) and check if both contains the same files, regardless of folders.

Is it possible?

This might sound like it has been asked before, but the solution 've seen is extracting folder contents in another folder, which is not efficient.

Best Answer

What I want to do is compare the SD directory and the media folder (each one has different structure) and check if both contains the same files, regardless of folders.

The fact that you want to compare the contents of, I'm assuming, a single folder on the SD card against another folder on the, e.g., Macintosh HD containing an entirely different hierarchical directory structure ... IMO a shell script is the better way to go.

The example shell script code, shown below, get saved as an executable and placed within the PATH passed to the shell in, e.g., Terminal, and does the following:

Creates a list of filenames on the SD Card containing the original files to loop through, attempting to find the duplicate files in the hierarchical directory structure of the directory designated as the parent directory containing the duplicate files on the e.g. Macintosh HD, regardless of where within the hierarchical directory structure they may be located.
If the original files do not exist somewhere within the parent hierarchical directory structure that supposed to contain the duplicate files, then the original files that don't already exist get copied to the parent directory that's supposed to contain the duplicate files.
If a filename exist in both locations, regardless of their hierarchical directory structure, a binary comparison is preformed, and if there is a difference it's reported.
- Any differences will need to be addressed manually.
- Note that because there may be a possibility that a file with the same name exists more than once in the parent hierarchical directory structure that supposed to contain the duplicate files, any reported difference in the binary comparison will need to be manually verified that the file on the SD Card also exists elsewhere in the local media folder.

In Terminal, assumes pwd is your Home directory, run the following compound command:

f="sdcheck"; touch "$f"; open -e "$f"; chmod 0755 "$f"

In the opened sdcheck document, copy and paste the example shell script code, shown below, into the document. Modify the value of dupdir variable to an existing local folder containing the duplicates from the SD Card. Then save and close the document.

^{Note: I'd move the sdcheck file to a location that's within the PATH passed to shell scripts, then you'd not have to preface the command with ./, or include it's fully qualified pathname.}

Back in Terminal:

% ./sdcheck
Parameter missing!... e.g.: /path/to/directory
Example: sdcheck '/Volumes/SD Card/Media' '/path/to/local_folder'
%

Example test:

% ./sdcheck '/Volumes/SD Card/Media' /Users/me/Pictures/SD Cards
NOTE: /Volumes/SD Card/Media/IMG_0004.JPG AND /Users/me/Pictures/SD Cards/Card 2/IMG_0004.JPG DIFFER!...
/Volumes/SD Card/Media/IMG_0005.JPG -> /Users/me/Pictures/SD Cards/IMG_0005.JPG
%

Example shell script code:

#!/bin/bash

if [ -z "${1}" ] || [ -z "${2}" ]; then
    echo "Parameter missing!... e.g.: /path/to/directory"
    echo "Example: sdcheck '/Volumes/SD Card/Media' /path/to/local_folder"
    exit 1
fi  

oridir="${1}"
dupdir="${2}"

if [ ! -d  "${oridir}" ]; then
    echo "The directory \"${oridir}\" does not exist!..."
    exit 1
fi

if [ ! -d  "${dupdir}" ]; then
    echo "The directory \"${dupdir}\" does not exist!..."
    exit 1
fi

while IFS= read -r orifile; do

    while IFS= read -r dupfile; do

        if [ -z "${dupfile}" ]; then
            cp -av "${orifile}" "${dupdir}"
        else
            if ! cmp -s "${orifile}" "${dupfile}"; then
                echo "NOTE: ${orifile} AND ${dupfile} DIFFER!..."
            fi
        fi

    done <<< "$(find "${dupdir}" -type f -name "${orifile##*/}")"

done < <(find "${oridir}" -type f ! -name '.DS_Store')

Using the example shell script code in Automator

To adapt the example shell script code for use in an Automator workflow, application and or Service/Quick Action, it needs to be modified with some additional code added, and is done so below.

The example Automator workflow uses AppleScript to choose the target folders, the SD Card and the local folder to be compared.

The example shell script code and example AppleScript code, shown below, was tested in Automator under macOS Catalina with Language & Region settings in System Preferences set to English (US) — Primary and worked for me without issue¹.

¹ _{Assumes necessary and appropriate setting in System Preferences > Security & Privacy > Privacy have been set/addressed as needed.}

The Automator workflow outline:

Run AppleScript action
Set Value of Variable action
- Variable: theSDCardMediaFolder
Run AppleScript action
- Options -- [√] Ignore this action's input
Set Value of Variable action
- Variable: theLocalMediaFolder
Get Value of Variable action
- Variable: theSDCardMediaFolder
- Options -- [√] Ignore this action's input
Get Value of Variable action
- Variable: theLocalMediaFolder
Run Shell Script action

The Automator actions, code, and settings:

In the first Run AppleScript action, replace the default code with the following example AppleScript code:

activate

set theMsg to ¬
    "In the next Choose Folder dialog, select the target folder on the SD Card..."

display dialog ¬
    theMsg buttons {"Continue"} default button 1 ¬
    with title "Select the SD Card Media Folder"

set theSDCardMediaFolder to ¬
    the POSIX path of ¬
        (choose folder with prompt ¬
            "Select the SD Card Media Folder:" default location "/Volumes")

set theSDCardMediaFolder to ¬
    characters 1 thru -2 of ¬
    theSDCardMediaFolder as string

return theSDCardMediaFolder

In the first Set Value of Variable action, set the value the variable to: theSDCardMediaFolder

In the second Run AppleScript action, replace the default code with the following example AppleScript code:

activate

set theMsg to ¬
    "In the next Choose Folder dialog, select the Local Media Folder."

display dialog ¬
    theMsg buttons {"Continue"} default button 1 ¬
    with title "Select the Local Media Folder"

set theLocalMediaFolder to ¬
    the POSIX path of ¬
        (choose folder with prompt ¬
            "Select the Local Media folder:" default location "/Users")

set theLocalMediaFolder to ¬
    characters 1 thru -2 of ¬
    theLocalMediaFolder as string

return theLocalMediaFolder

Options -- [√] Ignore this action's input

In the second Set Value of Variable action, set the value the variable to: theLocalMediaFolder

In the first Get Value of Variable action:

Variable: theSDCardMediaFolder
Options -- [√] Ignore this action's input

In the second Get Value of Variable action:

Variable: theLocalMediaFolder

In the Run Shell Script action:

Shell: [/bin/bash]
Pass input: [as arguments]

Replace the default code with the following modified example shell script code:

oridir="${1}"
dupdir="${2}"

logfile="/private/tmp/SD_Card_File_Check_Log.txt"

[ -z "${2}" ] && exit

echo "Begin: $(date -j)" >> "${logfile}"

while IFS= read -r orifile; do

    while IFS= read -r dupfile; do

        if [ -z "${dupfile}" ]; then
            cp -av "${orifile}" "${dupdir}" >> "${logfile}" 2>&1
        else
            if ! cmp -s "${orifile}" "${dupfile}"; then
                echo "NOTE: ${orifile} AND ${dupfile} DIFFER!..." >> "${logfile}"  2>&1
            fi
        fi

    done <<< "$(find "${dupdir}" -type f -name "${orifile##*/}")"

done < <(find "${oridir}" -type f ! -name '.DS_Store')

echo "End: $(date -j)" >> "${logfile}"
echo  >> "${logfile}"

open -e "${logfile}"

Notes:

For the use in Automator the workflow was saved as an application.

When the Automator workflow finishes, it opens the logfile in TextEdit for one's review. Here is a sample of what it could look like:

Begin: Thu May 27 14:29:59 EDT 2021
NOTE: /Volumes/SD Card/Media/IMG_0004.JPG AND /Users/me/Pictures/SD Cards/Card 2/IMG_0004.JPG DIFFER!...
/Volumes/SD Card/Media/IMG_0005.JPG -> /Users/me/Pictures/SD Cards/IMG_0005.JPG
End: Thu May 27 14:29:59 EDT 2021

I did not have an SD Card handy to review its hierarchical directory structure and as such I use a DMG image mounted and pathed as shown in the logfile output above.

The testing set of images was a very small sampling, nonetheless the code logic works as expedited as coded.

In the example Automator workflow image below, there is an obvious disconnect in the center between some of the actions. This is wanted and due to: Options -- [√] Ignore this action's input

_{Note: The example shell script code and example AppleScript code is just that and sans any included error handling does not contain any additional error handling as may be appropriate. The onus is upon the user to add any error handling as may be appropriate, needed or wanted.}

_{For AppleScript error handling... Have a look at the try statement and error statement in the AppleScript Language Guide. See also, Working with Errors. Additionally, the use of the delay command may be necessary between events where appropriate, e.g. delay 0.5, with the value of the delay set appropriately.}

Related Solutions

Mac – Automator: Issue with nested folders

The way to create nested folder structures is probably AppleScript, not Automator. I don't think Automator can create nested folder structures as easily as AppleScript.

The following AppleScript creates a folder structure of the form

- My first folder
  - A folder within my first folder
    - Another folder within both of those

within the frontmost folder in Finder. Here it is:

try
  tell application "Finder" to set FOLDER0 to (folder of the front window as alias)
end try

tell application "Finder"
  set folder1 to make new folder at FOLDER0 with properties {name:"My first folder"}
  set folder2 to make new folder at folder1 with properties {name:"A folder within my first folder"}
  set folder3 to make new folder at folder2 with properties {name:"Another folder within both of those"}
end tell

I've deliberately tried to keep it basic. All the structure is declared in the second tell block, and hopefully it should be easy to see how to extend it. If you want to test if before attaching it to a service, copy and paste it into AppleScript Editor (in the Utilities folder).

This script is fairly naive, and throws an error if the folders already exist. If that's a case you might be working with, then this question on Stack Overflow explains how to modify a script like this and only create folders that don't already exist.

If you want to create lots of folders with similar names (folder001, folder002, ..., folder999), then this forum thread on Apple Insider may be helpful.

You then want to bind this to a service in Automator (which I see from the question you know how to do). The service should receive folders in Finder, and you use the Run AppleScript action to run the script.

Once you've got that service installed, you can right-click on a folder in the Finder, and there'll be a service which creates this structure for you.

To get the different templates, I can think of two methods.

You could create multiple services ("Create nested folders 1", "Create nested folders 2", …). But that seems rather messy. Again, AppleScript is our friend. We can make a dialog that looks like this appear:

enter image description here

and which option we choose dictates which folder structure gets created. Here's the relevant code:

set choice to choose from list {"template1", "template2", "template3"}

try
  tell application "Finder" to set FOLDER0 to (folder of the front window as alias)
end try

if choice is "template1" then
  -- create some folders
end if

if choice is "template2" then
  -- create some different folders
end if

if choice is "template3" then
  -- create a third set of folders
end if

Replace the lines starting with -- with the folder generating code that we discussed above. The list in the first line gives you a list of choices, and then you create an if block for each choice that generates the appropriate folder structure.

If you use this AppleScript in Automator, then the procedure is exactly the same, except now you get the choice when you run the service in Finder.

Quick action opens automator

Your workflow receives No input in any application. Quick Actions are designed to run only on the type of file (or other object) that is currently selected. So I'm surprised that it shows up at all. Presumably Automator opens it because it "no input" is the default of an empty or unfinished workflow.

As your workflow doesn't require an input, (i.e. any given file as an argument), but just copies a specific file, then an application is a better choice.