Shell – Find a file and make a symlink to parent using find and -exec

command-substitutionfindquotingshell-script

I am trying to use find to find files matching a certain pattern, and then symlink their parent directorys to another directory, this is my current script (I'm doing this on mac so -printf won't work here):

#!/usr/bin/env bash

PROCESS_DIR="/Users/me/Google Drive/Me"
OUTPUT_DIR="/Users/me/OfflineFolder"

# Copy directory structure and then make symlinks
rm -R "$OUTPUT_DIR";
mkdir "$OUTPUT_DIR";
cd "$PROCESS_DIR";

find . -name *.md -exec ln -vs $(dirname {}) "$OUTPUT_DIR" \;

But it doesn't seem to be working. This script however does work (makes symlinks to all the md files on my computer:

# Copy directory structure and then make symlinks
rm -R "$OUTPUT_DIR";
mkdir "$OUTPUT_DIR";
cd "$PROCESS_DIR";
find . -type d -exec mkdir -vp "$OUTPUT_DIR/{}" \;
find . -name *.md -exec ln -vs "$(pwd)/{}" "$OUTPUT_DIR/{}" \;
find "$OUTPUT_DIR" -type d -empty -delete

Any idea why this is not working? I've tried several different approaches (including using find $PROCESS_DIRECTORY instead of cd $PROCESS_DIRECTORY). Thanks

Best Answer

You have two problems, both related to the order in which things happen.

find . -name *.md -exec ln -vs $(dirname {}) "$OUTPUT_DIR" \; is a single command. The shell parses it before executing it. Amongst the step in parsing:

  • $(…) is a command substitution, so dirname {} is executed, yielding . (there's no directory part in {}).
  • $OUTPUT_DIR is a variable substitution, so it's replaced by its value.
  • *.md is a glob pattern, so it's replaced by the list of matching files. If there are no matches, the pattern remains in place.

This completes the work required to determine what command to execute.

  • If there are no files matching *.md in the current directory then the following command is executed with the given arguments: find, ., -name, *.md, -exec, ln, -vs, ., /Users/me/OfflineFolder, ;.
  • If *.md matches bar.md and foo.md then the command is find, ., -name, foo.md, -exec, … (and find will miss any file called something-other-than-foo.md in subdirectories).
  • If *.md matches bar.md and foo.md then the command is find, ., -name, bar.md, foo.md, -exec, … (and find will complain of a syntax error).

You want to execute dirname on the find results. This means that you need to instruct find to run dirname. You can do that, but find doesn't have any mechanism to gather the output from dirname and pass it as an argument to ln. For this you need a tool such as a shell, where it's a command substitution.

So here's the strategy: tell find to invoke a shell, and tell that shell to run the command involving ln and dirname. You need to take care of quoting. Put the shell command in single quotes to avoid having its special characters interpreted by the outer shell. Also put the pattern for -name in quotes so that it's passed to find and not expanded by the outer shell.

find . -name '*.md' -exec sh -c 'ln -vs …' \;

The next step is to complete the …. Do not use {} inside the shell command: that would just place the file name as a snippet of shell code, and any special characters would be parsed by the inner shell. Instead, pass the file name given by find as an argument to the shell script. The first argument after sh -c CODE is the name of the shell instance ($0), but you can use it for whatever purpose you like; subsequent argument are the positional parameters ($1, $2, …).

find . -name '*.md' -exec sh -c 'ln -vs "$(dirname "$0")" "$1"' {} "$OUTPUT_DIR" \;

I've passed $OUTPUT_DIR as an argument to the script. It doesn't matter here because the value doesn't contain any shell special characters, but it's a good habit to get into, you never know when someone might change the path to e.g. include spaces. Another possibility would be to pass it through the environment:

export OUTPUT_DIR
find . -name '*.md' -exec sh -c 'ln -vs "$(dirname "$0")" "$OUTPUT_DIR"' {} \;

Instead of dirname, you can use a textual substitution: remove everything after the last slash. You don't have to worry about the special case where there's no directory part as this doesn't happen for a file name passed by find.

export OUTPUT_DIR
find . -name '*.md' -exec sh -c 'ln -vs "${0%/*}" "$OUTPUT_DIR"' {} \;

You can use the + form of -exec to speed things up a little. I pass _ as $0 and the subsequent arguments are the file names which the for loop iterates over.

export OUTPUT_DIR
find . -name '*.md' -exec sh -c 'for x; do ln -vs "${x%/*}" "$OUTPUT_DIR"; done' _ {} +
Related Question