zsh – How to cp Hidden Files with Glob Patterns

cpdot-fileswildcardszsh

Situation :

$ mkdir foo && touch foo/.test
$ cp foo/* .
zsh: no matches found: foo/*
(or bash : cp: cannot stat ‘foo/*’: No such file or directory)

I have a directory full of hidden folders and files. What is happening and what is the solution?

Best Answer

Disclaimer: This answer deals with Bash specifically but much of it applies to the question regarding glob patterns!

The star character (*) is a wildcard. There are a certain set of characters that it will take the place of and the first character being a dot (.) isn't one of them. This is a special case just because of how the Unix filesystems work, files that start with a dot are considered "hidden". That means that tools such as cp, ls, etc. will not "see" them unless explicitly told to do so.

Examples

First let's create some sample data.

$ mkdir .dotdir{1,2} regdir{1,2}
$ touch .dotfile{1,2} regfile{1..3}

So now we have the following:

$ tree -a
.
|-- .dotdir1
|-- .dotdir2
|-- .dotfile1
|-- .dotfile2
|-- regdir1
|-- regdir2
|-- regfile1
|-- regfile2
`-- regfile3

Now let's play some games. You can use the command echo to list out what a particular wildcard (*) would be for a given command like so:

$ echo *
regdir1 regdir2 regfile1 regfile2 regfile3


$ echo reg*
regdir1 regdir2 regfile1 regfile2 regfile3

$ echo .*
. .. .dotdir1 .dotdir2 .dotfile1 .dotfile2

$ echo .* *
. .. .dotdir1 .dotdir2 .dotfile1 .dotfile2 regdir1 regdir2 regfile1 regfile2 regfile3

$ echo .dotdir*
.dotdir1 .dotdir2

Changing the behavior?

You can use the command shopt -s dotglob to change the behavior of the * so that in addition to files like regfile1 it will also match .dotfile1.

excerpt from the bash man page

dotglob If set, bash includes filenames beginning with a `.' in the results 
        of pathname expansion.

Example:

$ shopt -s dotglob
$ echo *
.dotdir1 .dotdir2 .dotfile1 .dotfile2 regdir1 regdir2 regfile1 regfile2 regfile3

You can revert this behavior with this command:

$ shopt -u dotglob
$ echo *
regdir1 regdir2 regfile1 regfile2 regfile3

Your situation?

For you you're telling cp that you want to copy all the files that match the pattern *, and there aren't any files.

$ cp foo/.* .

Or you can do this if you want everything in the foo folder:

$ cp foo .

Or you can be explicit:

$ cp foot/.* foo/* .

A more compact form using brace expansion in bash:

$ cp foo/{.,}* .

At any time you can use the echo trick to see what your proposed file patterns (that's the fancy term for what the star is a part of).

$ echo {.,}*
. .. .dotdir1 .dotdir2 .dotfile1 .dotfile2 abc regdir1 regdir2 regfile1 regfile2 regfile3

Incidentally if you're going to copy a directory of files + other directories, you typically want to do this recursively, that's the -R switch to cp:

$ cp -R foo/. .
Related Question