Try this:
find . -type f ! -name "*.*" -delete
However, note that the above will not delete files whose name ends in a .
, for example foo.
. To delete those as well, use this instead:
find . -type f ! -name "*.?*" -delete
With bash
Assuming the disk in question is mounted under /media/disk1
:
$ shopt -s globstar dotglob
$ stat -c '"%n",%s,%y' /media/disk1/**/* >disk1.csv
shopt -s globstar dotglob
turns on recursive globbing feature of bash (enables use of '**', see https://unix.stackexchange.com/questions/49913/recursive-glob). It also turns on matching of files starting with a .
, als known as hidden files.
stat
is the program used to get file meta data. Basically this program will be run for each file on the disk.
-c '"%n",%s,%y'
specifies the output format for the stat command. %n
is the file name, enclosed in double quotes, %s
is the file size, %y
is the last modification time. (see stat --help
)
/media/disk1/**/*
tells bash to hand all the file names recursively found under that path to the pogram (stat), for both, normal and hidden files, since dotglob is enabled.
>disk1.csv
redirects output into a file named disk1.csv.
The output in disk1.csv will look like this for my home for instance:
$ stat -c '"%n",%s,%y' /home/seb/**/*
"/home/seb/111",82,2018-03-26 18:38:04.048099912 +0200
"/home/seb/app",4096,2017-07-13 23:39:06.509862769 +0200
"/home/seb/Applications",4096,2018-03-14 20:20:48.552005660 +0100
"/home/seb/Applications/arduino-1.8.2",4096,2017-05-29 20:45:01.184017517 +0200
"/home/seb/Applications/arduino-1.8.2/arduino",946,2017-03-22 13:32:41.000000000 +0100
[...]
I tested to import the resulting csv into libreoffice calc and it worked nicely, also with funny file names with line breaks in them. It will probably choke on file names with double quotes in them.
ARG_MAX
The above command will fail if the total number of files is too high or the total number of characters in all file names is too high. For small drives (USB thumb drives etc.) it should be enough, but if you are indexing a big disk with millions of files you would probably hit that limit.
You can run the following instead, it will produce the same output (and eat less memory):
find /media/disk1 -type f -print0 | xargs -0 stat -c '"%n",%s,%y' >disk1.csv
For the "find .. -print0 | xargs -0 .." pattern you will find many answers here already, e. g. Difference between "xargs" and command substitution?
Best Answer
Usually, you will want to just open Shotwell and drag in the folder.
If you need to, you can make the shell build a list of CR2 files and tell Shotwell to open just those files. Occasionally that may be what you want, but in this case that did not do quite what you needed due to Shotwell's behavior -- in some other circumstances, a feature -- of automatically showing you other image files (of all types) residing in the same folders as the images you opened.
To ensure Shotwell only knows about the images you are interested in, you could clone the directory tree somewhere else, copying only
.cr2
files and the folders that contain them, but nothing else. That may take up substantial additional space, though. So another approach is to construct a hierarchy of symbolic links -- where instead of having having a new directory tree of copies of your original.cr2
files, you have a new directory tree of syminks to them. This takes up far less space.This answer covers all those approaches.
Opening a Folder of Images (Including Subfolders) in Shotwell
Shotwell itself gives you two ways to open a whole directory tree of photos. If these will work for you, then you won't need to do anything clever in the command line or even use it at all. If not, you might end up doing this in connection with another technique. (For example, if you end up creating a new directory of symlinks or copies of your
.cr2
files, you can use one of these ways to open it.)Way 1: Just open the whole folder in Shotwell.
In Shotwell, in the File menu, click Import from Folder. Then select the folder you want to import images from. This won't be limited to
.cr2
files, but if you're opening files from multiple subfolders containing.cr2
images created from a digital camera, then most likely those will be the only files present.See Jakob's answer to Show all pictures recursively (include any subfolders) for more information.
Way 2: Drag the folder into Shotwell.
With most desktop environments and file browsers, you can also open Shotwell and drag the folder that contains your images into the Shotwell window. Like Import from Folder, this is another way Shotwell recommends to open photos.
When you drag from a file browser window or your desktop into Shotwell, you can drag to a specific library. Or if you don't have multiple libraries set up, you can just drag basically anywhere and it works. (You don't even have to drag into the left pane.)
This method is very convenient, and it is as powerful as manually selecting the folder (with File > Import from Folder, as described above). When you open files or folders using either of these methods, Shotwell gives you the option to open the photos without copying them into its library.
To import without copying, click Import in Place instead of Copy Photos.
Launching Shotwell with a List of Image Filenames
If for some reason you want or need to do this by running a command in the Terminal, you can run the
shotwell
command and pass each image's filename to it as an argument. Globstar makes it easy:That opens the
.cr2
files located anywhere in the current directory. It differs from running Shotwell and then dragging the folder in, in that Shotwell will only try to import the the files whose names you give it. However, since Shotwell often shows other image files from the same folders as the files you've told it to open, this may not be adequate for your needs. (And further information you've given about your problem suggests that it is not. See below for an alternative.)This does not include files or subfolders whose names start with
.
. However, if you have CR2 files whose names start with a dot, then most of the time you probably don't want to open them in Shotwell anyway. Unless you've deliberately named your files or folders in this way, a dotfile with an image or document filename extension is more likely to be a metadata file rather than an actual image or document. For example, macOS often adds such files (e.g.,._kittens.cr2
for an actual picturekittens.cr2
) when accessing external media (including flash drives and digital cameras).If you do want to also find files whose names start with
.
or that are in folders whose names start with.
, then enable dotglob as well. You can do this withshopt -s dotglob
or by passing bothglobstar
anddotglob
to theshopt
builtin:By default, globs are case-sensitive. But you might want to match filenames case-insensitively, so as to open files ending in
.cr2
,.Cr2
,.cR2
, and.CR2
. To do that, either:Enable nocaseglob by running
shopt -s
withnocaseglob
, either by itself or with other shell options. For example, to runshotwell
on all.cr2
files, searching case-insensitively and including.
-files and files in.
-directories:Or, use
.[cC][rR]2
instead of.cr2
in your glob pattern itself. For example, this accomplishes the same as above:Note that case sensitivity is entirely separate from whether or not
.
files and folders are matched. You can removedotglob
from the examples above and they can still be case-insensitive.To specify a directory rather than just using the current directory, put its name and a
/
before the**/*.cr2
glob. For example, replacing theshotwell
command above with this one will open all the.cr2
files inside thePictures
subdirectory of your home directory:Unlike with the first command that uses your current directory, no matter where you run that command you will be telling Shotwell to open the same files.
Testing
If you really have to use the terminal for this, then I'm guessing it may be part of something more complex. If you like, you can test such commands before actually running them by replacing
shotwell
withprintf '%s\n'
. Instead of passing the filenames to Shotwell, that lists them one per line. For example, on my machine:Turning Off Globstar and Other Shell Options
The globstar shell option has be enabled to work, which is what the
shopt -s globstar
command (above) did. If it's already enabled, then you don't need multiple commands. And you only need to set globstar once per shell instance.You can turn globstar back off by running:
And you can check whether globstar is on or off by running:
If you're using globstar in a script, you should consider turning it back off if there might be code that comes afterwards in the same script that is not written with globstar in mind. For example, writing
**
when you mean*
is usually no problem when globstar is disabled, but with globstar turned on, it can cause serious problems by making a command affect far more files than someone intended.Similarly, if you used dotglob or nocaseglob and you want too turn them off, you can do so with
shopt -u dotglob
andshopt -u nocaseglob
, respectively. You can also pass multiple shell options toshopt -u
to unset them at the same time. For example, to turn offglobstar,
dotglob
, andnocaseglob
:Compared to globstar, there are fewer serious mistakes you might make because you forgot you left dotglob or nocaseglob on. But other people (including people you consult on sites like Ask Ubuntu) usually assume dotglob and nocaseglob are disabled when they suggest commands for you to use, unless you tell them otherwise.
For more information on these glob-related shell options (and also on globs in general), see the bash reference manual. They are all covered primarily in 4.3.2 The Shopt Builtin. For globstar, see also 3.5.8.1 Pattern Matching. For dotglob, see also 3.5.8 Filename Expansion.
Building a New Directory Tree
Since in your case you seem to really need Shotwell to show only
.cr2
files, you have non-.cr2
files in the same folders with.cr2
files, and image browsing in Shotwell is automatically offering those files too, it looks like you will need to have a folder that contains no other image files than the.cr2
files you're trying to open.Moving the files, as you've suggested, is one approach, but I recommend instead either copying the files or, better, making symbolic links to the files (see also FOLDOC), because:
There is another decision for you to make, too. Do you create all the new entries (whether they be copies or symlinks) in the same directory, flattening the file hierarchy? Or do you create directories at the destination corresponding to each of the directories at the source that contain at least one
.cr2
file?Flattening the file hierarchy lets you see all the files at once, easily. In particular, it is easy to see the files' names and sizes all at once, as well as to select any combination of the files in a graphical file browser.
But it also loses information about what folders they were in, which may be valuable (though you can return to the original folder and look for them there if you don't move the files).
The bigger problem with flattening the file hierarchy, though, is what happens if you have two or more files of the same name in different places. If you flatten the file hierarchy, then depending on how you do it, either all but the first will fail to be copied (or linked) and you'll get an error or warning, or all but the last copy (or link) will be overwritten.
This is another reason I recommend copying or making symlinks rather than moving files. If you "successfully" move multiple files into the same file, you've lost all but the last one!
Because of the disadvantages of flattening everything into a single directory without subdirectories, the approaches I take here build a new directory tree at the destination.
Way 1: Copying
I show this first because it's a little simpler, but I suggest you instead create symbolic links ("Way 2"), especially if the destination drive might not have enough space to hold all the files. "Way 2," which is a bit more complicated, is much more detailed because there seem to be fewer resources on how to use that technique, it's the way I expect most people in this situation will prefer, and information that applies to both methods -- basic advice about escaping spaces in paths -- is given there rather than here.
When you need to copy a directory tree and you need to customize the copy operation, you should usually use
rsync
instead ofcp
. This is one of those cases.Replacing
src
with the path to the top-level directory containing all your CR2 files, and replacingdest
with the name of the directory you want to create that will contain a hierarchy of symbolic links to the CR2 files, run:Don't forget the
/
after the name of the source directory, unless you want an additional top-level directory to be created insidedest
. That is, if you omit the trailing/
, thenrsync
will make the destination directory for you, insidedest
. For example, supposesrc
is a relative path containing no/
. Then:src/
in yourrsync
command, you get a symlinkdest/foo/bar.cr2
for the filesrc/foo/bar.cr2
.src
in yourrsync
command, you get a symlinkdest/src/foo/bar.cr2
for the filesrc/foo/bar.cr2
.Since it could be confusing for the top-level directory where symlinks will be placed to have the same name as that of the top-level directory containing the
.cr2
files that the symlinks will point to, I recommend you include the/
at the end of the destination directory name in yourrsync
command. (And, for the source directory, give it a directory that is empty, doesn't exist, or is otherwise dedicated to this purpose. Otherwise, you'll be cluttering up some other folder.)As is generally the case when you pass the name of a file or directory to a command-line program or script, if your destination or source directory paths contain any blank spaces (or characters treated specially by the shell, like
\
,$
,'
, and various other punctuation), then you will have to quote them. I've detailed how to do this, for the most common case of spaces, in "Way 2"--the advice about quoting in that section applies here as well. However, if you're not sure how to quote the name of a folder, you can have Ubuntu do it for you by simply dragging the folder into the Temrminal; its full path will be pasted in with correct quotation.After copying just the
.cr2
files and the folders that contain them, you can open the destination folder up in Shotwell. (See "Opening a Folder of Images (Including Subfolders) in Shotwell" above.) However, before runningrsync
, I recommend you decide if want to skip entries that start with.
and if you want to include entries with case-variant suffixes. If so, you'll have to run some slightly differentrsync
command.Customizing the Handling of Dotted Entries and Case Sensitivity
You may want to use a different
rsync
command to change whether or not files and directories that start with a dot are skipped and whether or not you want case-sensitivity in matching the suffix.cr2
.The command above does include directories and files whose names start with
.
. To exclude such files and directories, put--exclude='.*/' --exclude='.*'
before all other--include
s and--exclude
s. That is, use thisrsync
command instead (with the appropriate substitutions forsrc
anddest
, of course):The
rsync
commands shown thus far have been case-sensitive and will not copy files that end in.CR2
,.Cr2
, or.cR2
rather than.cr2
. If you want such files to be copied, then you'll need to account for that by using the--include
pattern*.[cC][rR]2
in place of*.cr2
.This third command is case-insensitive and (like the first command) does not skip files and directories whose names start with
.
:If you take the approach of copying files with
rsync
, and you're not sure which of these commands you want, I suggest using that one if you want to make sure not to miss anything. However, you should be aware that dotfiles may contain metadata, and dotfiles named like images may not really be images. See the discussions on dotfiles in the "Launching Shotwell with a List of Image Filenames" section above, and on dotfiles and case sensitivity in "Way 2" below.This fourth command is case-insensitive and (like the second command) does skip files and directories whose names start with
.
:Way 2: Creating Symbolic Links
This is what I recommend you do, though Way 1 is okay if you have enough space on the destination drive. You don't have to read Way 1 to use this.
The method presented here makes a list of
.cr2
files and loops through the list, creating target directories as needed and populating them with symbolic links to each.cr2
file.This is just complicated enough that I suggest putting it in a script rather than just typing it in interactively. You can put the script wherever you like, but for convenience, I'll show how to do it with the script placed directly in your home folder.
Write (or paste in) the script.
Create a text file,
make-symlinks
, in your home folder. No file extension is needed.You can do this with any text editor, like Vim, Nano, Gedit, or Leafpad. Don't use a word processor like LibreOffice Writer, though. (You could even do it by running
cat > ~/make-symlinks
, pasting in the contents of the file, and pressing Ctrl+D on a new line, if you wanted.)If you name the script something other than
make-symlinks
, make sure to use that name when you run commands that use the script later.Put that in the file, and save the file. This script uses globstar, a Bash feature explained above in "Launching Shotwell with a List of Image Filenames".
You can change how it handles dotted entries and case sensitivity.
Depending on your needs, you may want to make some changes:
If you don't want to include
.cr2
files whose names start with.
or that reside in directories whose names start with.
, removedotglob
from theshopt
line.Files and directories whose name start with
.
are often used for metadata (see the "Launching Shotwell with a List of Image Filenames" section above). On the other hand, if you want to make sure you have everything, or if there's something inexplicably missing that you are pretty sure is there, then should keepdotglob
.If you want to include only files that end in exactly
.cr2
but not case variants (.CR2
,.Cr2
, and.cR2
), then you should removenocaseglob
. Since your situation involves data recovery and Shotwell will not show you any of the files you missed when you open up the directory of symbolic links -- that's the point of this method, after all -- you will likely want to keepnocaseglob
.Make it executable.
Run this command to mark your script executable:
Run it at the source and pass the destination as an argument.
Then change directory to the source directory, i.e., the top-level folder that contains the pictures. For example, if they are in the
Pictures
subdirectory of your home directory, you would run:This is where you'll run the script. Pass the destination directory to the script as a command-line argument.
For example, if you want a directory called
cr2s
to be created on your Desktop to contain the symbolic links, run:Or if you just want that directory directly in your home folder:
Or if you have a drive mounted at
/media/you/MyDrive
and you want to create a folder on it calledlinks
to contain the symbolic links:But if the drive is
/media/you/My Drive
(with a space) then you must use quotation. Any of these works (use just one, don't run them all):But if you use
~
for your home directory, don't quote that:make-symlinks
creates the destination directory for you if it doesn't already exist, just as it creates subdirectories within it. However, it still works if the destination already exists.You can even run it multiple times and pass the same destination directory (for example, if you interrupted it with Ctrl+C), though you'll get
failed to create symbolic link
errors for the symlinks that already exist, and that could be confusing.You probably don't want to pass the name of a directory that already exists and contains unrelated files (like your home directory) to
make-symlinks
. If you do, you can expect a whole bunch of symlinks and subfolders to be created in it. You probably want to use a new directory -- or one that doesn't exist yet -- formake-symlinks
.Import the newly created symlink tree into Shotwell.
After you run
make-symlinks
, just import the directory you passed to it into Shotwell, using either of the methods given in the first part of this answer ("Opening a Folder of Images (Including Subfolders) in Shotwell"). For example, you could open Shotwell and drag the folder in.Since all the the entries in folder are either symbolic links to
.cr2
files or directories that contain them, with no other file types, Shotwell should not see any of the other files.