Bash – How to find files in directories listed in a file

bashdirectory-structurefind

In a directory there are several subdirectories, something like:

TopLevelDir
+-- SubDir1
+-- SubDir2
+-- SubDir3
+-- SubDir4
+-- SubDir5
+-- SubDir6

I need to find files containing some specific text in a specific subset of those subdirectories. What is needed can be accomplished with:

find SubDir1 SubDir2 SubDir4 -type f -exec grep -H "desired text" {} \;

A similar search will be needed frequently. To save time and typing, I would like to store the names of the subdirectories to be searched in a file and use that when I run the find command next time, something like:

find subdirs2search.txt -type f  -name="*.txt" -exec grep -H "desired text" {} \;

Searching the web, checking man pages, and even checking my *nix book hasn't turned up anything on how to do this.

Is this possible? If so, how can it be done?

Best Answer

If the directory names are one-per-line, then you could avoid issues with directories that have spaces or tabs or wildcard characters in their names by using readarray (bash v4+):

readarray -t dirs < subdirs2search.txt
find "${dirs[@]}" ...

That would still not help if some directory names start with -, but with GNU find, there's no way around that.

Sample data

And I had sample directories with files in them like this:

$ mkdir -p dirs/{1..5}
$ touch dirs/{1..5}/afile

Resulting in this:

$ tree dirs/
dirs/
|-- 1
|   `-- afile
|-- 2
|   `-- afile
|-- 3
|   `-- afile
|-- 4
|   `-- afile
`-- 5
    `-- afile

Example run

Now if we run this command against this tree:

$ find dirs/ -type f -print0 | fgrep -zFvf ./ignore.txt
dirs/5/afiledirs/4/afiledirs/3/afile

We can see that we're only getting back the files that are in directories not listed in ignore.txt.

So we can add a rm to the end to remove the non-excluded files.

$ find dirs/ -type f -print0 | fgrep -zFvf ./ignore.txt | xargs -0 rm -f

Checking we can see that it worked:

$ tree dirs/
dirs/
|-- 1
|   `-- afile
|-- 2
|   `-- afile
|-- 3
|-- 4
`-- 5

Problems to be worked out

One big problem with this approach is that the strings in the ignore.txt file might match other portions of the directory structure. So some care needs to be paid to making sure that the strings in this file are unique in the way that you expect.

Some blocking could be put around the strings so that they're anchored to the beginning or the end of the string to protect them.

Details

The above commands are doing the following:

finding all the files under the directory dirs
filtering out any files that are under a directory present in the igonre.txt file
passing the filter list via xargs to the rm -f command

Best Answer

Related Solutions

How to “find” both regular files and directories

Bash – Delete all files in directories except those whose path are listed in a file

Sample data

Example run

Problems to be worked out

Details

Related Question