Bash – grepping with “shopt -s globstar extglob”

bashwildcards

I have shopt -s globstar extglob in my ./bashrc. I'm searching for files that refer to string Foo in the following command:

grep -Fwn Foo /**/src/**/*.@(h|cpp)

Now I want to except all refers from files Foo.h and Foo.cpp. The following is the command that gain this goal

grep -Fwn Foo /**/src/**/*.@(h|cpp) | grep -v Foo.h | grep -v Foo.cpp

I'd like to make it without piping command using ! and & in location pattern. I mean something like the following

grep -Fwn Foo /**/src/**/@(*.@(h|cpp)&(!(Foo.*)))

Why the command from above do not work and how to fix it?

Best Answer

Simply because there's no such thing as a &(...) operator in bash. bash only implements a subset of ksh patterns with extglob. Here you want:

grep -Fwn Foo /**/src/**/!(Foo).@(h|cpp)

With ksh93, you can use & this way:

grep -Fwn Foo /**/src/@(*.@(h|cpp)&!(Foo*))

zsh has a and-not operator with extendedglob:

grep -Fwn Foo /**/src/(*.(h|cpp)~Foo*)

Related Solutions

bash – Passing Braces as Arguments to Bash Function

When you run findinfiles {*.js,*.html,} {release,dev,} "span", here is what happens:

Bash parses quotes and splits the command into words: findinfiles, {*.js,*.html,} {release,dev,} span.
Bash expands the braces, resulting in the list of words findinfiles, *.js, *.html, release, dev, span.
Bash expands the wildcard patterns *.js and *.html to the list of matching file names. If no file name matches either pattern, it is left alone.
Bash looks up the name findinfiles, finds out that it's a function, and executes the function with the supplied parameters.

You can prevent the braces and wildcards from being expanded at the function call, but then the braces will appear literally in the arguments, which isn't particularly useful.

I suggest changing the format of your function call. Instead of using braces, use only commas, and manually split the arguments at commas inside the function.

findinfiles () {
  local patterns="$1" excludes="$2" pattern="$3"
  shift 3
  typeset -a cmd dirs
  if [[ $# -eq 0 ]]; then dirs=(.); else dirs=("$@"); fi
  cmd=(grep --color=auto -iRnHr)
  local IFS=,
  for x in $patterns; do
    cmd+=("--include=$x")
  done
  for x in $excludes; do
    cmd+=("--exclude-dir=$x")
  done
  "${cmd[@]}" "${dirs}"
}

Explanations:

Store the first three parameters in local variables. Any extra parameters are directories to search in.
Set dirs to the list of extra parameters. If there are none, use . (current directory) instead.
Set IFS to a comma. When a script contains an unquoted variable expansion like $patterns and $excludes above, the shell performs the following:
1. Replace the variable by its value.
2. Split the variable into separate words wherever it contains a character that is present in IFS. By default, IFS contains whitespace characters (space, tab and newline).
3. Treat each word as a wildcard pattern and expand it to the list of matching files. If there is no matching file, leave the pattern alone.
(To avoid these expansion steps, use double quotes around the variable substitution, as in patterns="$1" and so on in the script above.)
The function builds the command line to execute incrementally in the array variable cmd. At the end, the command is executed.

Alternatively, you could build on the following settings:

shopt -s extglob globstar
fif_excludes='release dev'
alias fif='grep --color=auto -inH $fif_excludes'

Run commands like

fif span **/*.@(js|html)

Explanations:

shopt -s extglob activates the @(js|html) form of wildcard pattern, which matches either js or html. (This option activates other pattern forms, see the manual for details.)
shopt -s globstar actives the pattern **/ which matches subdirectories at any depth (i.e. it performs a recursive traversal).
To change the exclude list (which I expect doesn't happen often), modify the fif_excludes variable.

Bash copy all files that don’t match the given extensions

You can use find to find all files in a directory tree that match (or don't match) some particular tests, and then to do something with them. For this particular problem, you could use:

find -type f ! \( -iname '*.png' -o -iname '*.gif' -o -iname '*.jpg' -o -iname '*.xcf' \) -exec echo mv {} /new/path \;

This limits the search to regular files (-type f), and then to files whose names do not (!) have the extension *.png in any casing (-iname '*.png') or (-o) *.gif, and so on. All the extensions are grouped into a single condition between $ ... $. For each matching file it runs a command (-exec) that moves the file, the name of which is inserted in place of the {}, into the directory /new/path. The \; tells find that the command is over.

The name substitution happens inside the program-execution code, so spaces and other special characters don't matter.

If you want to do this just inside Bash, you can use Bash's extended pattern matching features. These require that shopt extglob is on, and globstar too. In this case, use:

mv **/!(*.[gG][iI][fF]|*.[pP][nN][gG]|*.[xX][cC][fF]|*.[jJ][pP][gG]) /new/path

This matches all files in subdirectories (**) that do not match *.gif, *.png, etc, in any combination of character cases, and moves them into the new path. The expansion is performed by the shell, so spaces and special characters don't matter again.

The above assumes all files are in subdirectories. If not, you can repeat the part after **/ to include the current directory too.

There are similar features in zsh and other shells, but you've indicated you're using Bash.

(A further note: parsing ls is never a good idea - just don't try it.)

Best Answer

Related Solutions

bash – Passing Braces as Arguments to Bash Function

Bash copy all files that don’t match the given extensions

Related Question