Bash – Shell wildcards and dot files

bashshellwildcards

Why do the following two codes work as expected:

ls -d .[!.]?*
echo [D]*

But the following 2 don't:

ls -d [.][!.]?*
echo [D]

In the first command, I get an error:

ls: cannot access [.][!.]*: No such file or directory

When I am trying to get the same listing ls -d .[!.]?*. And for the second, the output is:

[D]

When I'm expecting an error proclamation along the lines of No such file or directory. What am I missing? What exactly makes an expression a wildcard, if those wildcard elements shown in the second set of examples above don't cut it?

Clarification (Also in comments):

The wildcard [D]* doesn't only output D, it also outputs Desktop, Downloads … etc. However, I also tried echo [D] when I had a file named D and when I didn't. The output worked when the file D was there, but I also got the output [D] when it wasn't. I don't get why. Why did the presence of the file in the directory change the expression [D] from being a wild card to not?

Best Answer

By default, bash (the shell) doesn't expand a filename beginning with a dot (considered a hidden file) unless the dot is explicitly specified. Since [.] is a pattern, rather than a literal character, it doesn't trigger the display of hidden files and so ls -d [.][!.]?* doesn't match anything.

You can change this default behavior by running shopt -s dotglob, which sets the option to always includes hidden files when expanding filename patterns (known as "globbing").

Example (the dollar sign at the beginning of the line indicates the shell prompt; don't type it in):

$ touch .foo
$ ls -d .[!.]?*
.foo
$ ls -d [.][!.]?*
ls: cannot access [.][!.]?*: No such file or directory
$ shopt -s dotglob
$ ls -d [.][!.]?*
.foo

The issue with echo [D] is that expansion doesn't happen if no filename is matched. Since you didn't have a file named D when you tried it, echo was given, and echoed, what you literally entered.

An example of pathname expansion only occurring when a filename matches:

$ touch apple
$ echo a*
apple
$ rm apple
$ echo a*
a*

If you delete all of the files which match [D]*, then echo [D]* will give you [D]*.

Related Solutions

Bash – find and globbing (and wildcards)

Both the find command and the shell are capable of file globbing.

This is unusual - most commands are NOT capable of globbing and rely entirely on the shell to expand globs. But find is a super duper mega power user tool that you can very easily hurt yourself with!

Example: when you do the command

   find /path -iname *.txt

The first thing that happens is the shell attempts to find all files that match *.txt in the current directory. If it finds any, it substitutes the names of all the matching files for the glob, and then calls the find command. The find command never sees the glob if this happens, the shell has expanded it out of existence.

But if there are no files in the current directory that match the glob, the shell shrugs its metaphorical shoulders and passes the glob unchanged to find. So at that point, the find command (which understands globs, remember) will output the names of all files it finds under /path that match the glob.

So using globs this way means that find will behave differently depending on the content of the current directory. This is almost certainly not what you want!

To prevent the shell from tampering with globs before find sees them, escape the globs with appropriate shell metacharacter quoting. Usually this just means putting your globstrings in single quotes like this

   find /path -iname '*.txt'

Remember, GLOBS ARE NOT REGEXES - the glob ".*" and the regex ".*" are very different and do not match the same strings!

Shell – Differing outputs of the commands “ls .” and “ls .”

It's because * doesn't match files starting with a . by default. Consider the following directory:

$ ls -la 
total 8404
drwxrwxrwx   2 terdon terdon 8105984 Dec 31 13:14 .
drwxr-xr-x 153 terdon terdon  491520 Dec 30 22:32 ..
-rw-r--r--   1 terdon terdon       0 Dec 31 13:14 .dotfile
-rw-r--r--   1 terdon terdon       0 Dec 31 13:14 file1
-rw-r--r--   1 terdon terdon       0 Dec 31 13:14 file2
-rw-r--r--   1 terdon terdon       0 Dec 31 13:14 file3.

Let's see what each of the globs you used expands to:

$ echo .*
. .. .dotfile

$ echo *.
file3.

$ echo *
file1 file2 file3.

As you can see, the * does not include files or directories starting with . so both ./ and ../ are ignored. The same thing happens with your ls example. In bash, you can change this with the dotglob parameter, which will cause * to expand to dotfiles as well (but not to . or .., those are still ignored):

$ shopt -s dotglob
$ echo *
.dotfile

Other shells behave differently. For example zsh also ignores the . and .. when using .*:

% echo .*
.dotfile

Best Answer

Related Solutions

Bash – find and globbing (and wildcards)

Shell – Differing outputs of the commands “ls .*” and “ls *.”

Related Question

Shell – Differing outputs of the commands “ls .” and “ls .”