Why Does `find` Access .gvfs Despite `-prune`?

findshell

_{beforehand: this evolved from a general question (I thought it was the case with every folder) to this specific corner case.

as it seemed that a few other people are as curious as myself, I decided to not delete this question but rework it.}

Let's say I want to list all items contained direct in my home folder, except the dot files & folders. IMHO, the following find command should do this (I explicitly wrote "dot items", because if you use a .hidden file the "hidden items" may differ):

find $HOME/ -mindepth 1 -maxdepth 1 -path "$HOME/.*" -prune -o -print

-mindepth 1 to ensure that $HOME itself is not included, -maxdepth 1 to limit to one folder level, -path "$HOME/.*" -prune to exclude everything in the home folder starting with a dot and -o -print to ensure that not the excluded elements are in the output according to the man page.

So far, so good. It kind of works (even if there are dot-folders for which you don't have the permissions), but it starts to choke on $HOME/.gvfs although it shouldn't be accessed at all due to -path ... -prune:

/home/user/file a
/home/user/folder b
...
find: '/home/user/.gvfs': Permission denied
/home/user/.gvfs
...
/home/user/file z

I know I can use -perm at find and 2> /dev/null to work around it, but what really appalled me is, that /home/user/.gvfs is included in the list (although all other dot items are excluded)!!!
and I'm pretty sure some not so careful people are tempted to get rid of the error with 2> /dev/null and move on. …if they recognize the error at all, because at longer lists with a few 100 lines you have to explicitly check for errors with something like adding 1> /dev/null prior to working with that command.

really ignoring a directory/tree?

Why is find trying to access /home/user/.gvfs at all, despite -maxdepth 1 or at least -prune should prevent this? e.g. from the man page:

‘-prune’ action (which only prevents further descent […])

even in an older man page (installed on my system) for -path it is stated:

To ignore a whole directory tree, use -prune rather than checking every file in the tree.

dangerous listing of an "ignored" directory itself!

If all /home/user/.* would be really ignored, perhaps the second part of my question won't be necessary, as it seems to me this is because of an internal error:
why is /home/user/.gvfs still in the list and how to prevent it? Is there a way without fiddling around with -user/-perm and 2> /dev/null for every find command (at least with a -prune action)?

future usage -> efficiency?

What is really ignored with -prune (or at least with -path ... -prune)?

~~If with -prune still (nearly) all files are checked, I would ditch using this complicated option and rather use ! or -not, especially as -prune is apparently much more error-prone.~~

Best Answer

This appears to be a bug in the GNU implementation of find. Here, the trigger seems to be the fact that the lstat() / fstatat() system call (to retrieve a file's metadata) fails with EACCESS (permission denied) on that .gvfs file.

That typically happens on Linux for files that are mount points of FUSE-based file systems for which the -o allow-other option has not being used. That can also happen with find -L / find -follow when there are symlinks that point to inaccessible areas.

I find here that -prune for those files returns false which is clearly a bug as -prune is documented (and required by POSIX) to always return true.

As a work around until that bug is fixed, you can replace -prune with '(' -prune -o -true ')' (note that -true is a GNU extension; a portable equivalent could be ! -name '' which is guaranteed to be true, though you could also do '(' -prune -o ! -prune ')')

Now, there are a few issues with your code itself. In:

find $HOME/ -mindepth 1 -maxdepth 1 -path "$HOME/.*" -prune -o -print

In most Bourne-like shells, that $HOME/ should be quoted as otherwise the $HOME expansion is subject to split+glob. You could also write it ~/.
-prune is to tell find not to descend into a directory. Here it's superfluous as maxdepth 1 will stop find from descending into any directory (other than the top-level (depth 0) one $HOME/).
-path takes a pattern. That means that the contents of $HOME will be taken as a pattern. If your $HOME is for instance /home/[112] me, -path '/home/[112] me/.*' would match on /home/2 me/.foo, but not on /home/[112] me/.foo. Also matching on the full path here is not necessary, you can match on the file name instead with -name '.*'.
* with GNU find only matches on characters, so .* matches on filenames that start with . and are also made of sequences of bytes that form valid characters. So you can only use that to match hidden files in locales where every sequence of byte form valid characters (such as the C locale).

So here, if the point is to list the non-hidden files in your HOME directory with their full paths, you'd do:

LC_ALL=C find ~/ -mindepth 1 -maxdepth 1 ! -name '.*'

Which would also avoid that GNU find bug.

Or the standard equivalent (though adds a /./ in the paths):

LC_ALL=C find ~/. ! -name . -prune ! -name '.*'

(here, the GNU find bug would be hit for files that do not start with . and are not lstat()able, but you'll see at that bug description above that whether non-lstat()able files are reported or not varies greatly with the implementation).

Here, I'd just use zsh's:

print -rC1 ~/*(ND)

Or if you don't want them sorted like with find:

print -rC1 ~/*(NDoN)

Related Solutions

Expected behavior of `find -depth` if execute permission denied for subdirectory

Probably the better approach is to handle such directories explicitly. I don't know whether these are standard features but at least with Gnu find this is possible:

find . \( -type d \( \! -executable -or \! -readable \) \) -prune -or -type d

of course, it is also possible to print a message if a directory is ignored:

find . \( -type d \( \! -executable -or \! -readable \) \) \
  -printf "Permission denied: %p\n" -prune -or -type d -print

POSIX-compliant recursive grep with no errors for inaccessible directories

To make grep print only the file name, pass the -l option. To search for a substring rather than a regular expression, pass the -F option.

To search recursively for files whose name matches a certain pattern, use find with the -type f and -name PATTERN primaries. Use -exec to invoke grep.

find . -name '*.sas' -type f -exec grep -F -l 'Carhart' {} +

If you want to avoid errors from directories that you aren't allowed to traverse, you can either use -perm, -user and -group to analyze permissions (which is difficult to get right, and won't work if you have ACL), or call test (which is slowed because it's an external program, but is more reliable).

find . -type d ! -exec test -r {} -a -x {} \; -prune -o \
       -name '*.sas' -type f -exec test -r {} \; -exec grep -F -l 'Carhart' {} +