Bash – Escape unknown characters from string for -exec

bashdirectoryescape-charactersfilesfind

I have a find command that finds certain files and directories. That find command then runs rsync with the files and directories found previously as source. The problem is that those files and directories can have all sort's of characters such as single and double quotes not to mention illegal characters from Windows etc…

How can I dynamically escape a string for use in rsync or other commands?

This command works by hard coding double quotes for rsync source string, but it will break if the string has double quotes in it.

find "/mnt/downloads/cache/" -depth -mindepth 1 \( \
-type f \! -exec fuser -s '{}' \; -o \
-type d \! -empty \) \
\( -exec echo rsync -i -dIWRpEAXogt --numeric-ids --inplace --dry-run '"{}"' "${POOL}" \; \)

resulting output:

rsync -i -dIWRpEAXogt --numeric-ids --inplace --dry-run "test/this " is an issue" /mnt/backing

Working command after info in answers applied:

find "/mnt/downloads/cache/" -depth -mindepth 1 \( \
                             -type f \! -exec fuser -s {} \; -o \
                             -type d \! -empty \) \
                             \( -exec rsync -i -dIWRpEAXogt --remove-source-files-- "${POOL} \; \) \
                             -o \( -type d -empty -exec rm -d {} \; \)

Best Answer

Your quoting problem is coming from you trying to solve a problem you don't have. Needing to quote arguments only comes into play when you're dealing with a shell, and if find is calling rsync directly, there is no shell involved. Using visual output isn't a good way to tell if it works or not because you can't see where each argument begins and ends.

Here's what I mean:

# touch "foo'\"bar"

# ls
foo'"bar

# find . -type f -exec stat {} \;
  File: ‘./foo'"bar’
  Size: 0           Blocks: 0          IO Block: 4096   regular empty file
Device: fd00h/64768d    Inode: 1659137     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1004/ phemmer)   Gid: ( 1004/ phemmer)
Access: 2017-12-09 13:21:28.742597483 -0500
Modify: 2017-12-09 13:21:28.742597483 -0500
Change: 2017-12-09 13:21:28.742597483 -0500
 Birth: -

Notice that I didn't quote the {} in the arg to stat.

Now that said, your command is going to be very non-performant, because you're calling rsync for every single matching file. There are 2 ways you can solve this.

As others have indicated you can use pipe the file list to rsync on stdin:

# find . -type f -print0 | rsync --files-from=- -0 . dest/

# ls dest/
foo'"bar

This will use null bytes as the file name delimiter since files can't contain null bytes in their name.

 

If you're using GNU find, you have another method of invoking -exec, and that's -exec {} +. In this style find will pass more than one argument at a time. However all the arguments are added to the end of the command, not in the middle. You can address this by passing the arguments through a small shell:

# find . -type f -exec sh -c 'rsync "$@" dest/' {} +

# ls dest/
foo'"bar

This will pass the list of file to the sh which will then substitute them in for the "$@"

Related Question