Remove characters from file names recursively

rename

I have hundreds of directories, some nested in other directories, with tens of thousands of files. The files need to have a date/time stamp removed from them.

An example filename is Letter to Client 27May2016~20160531-162719.pdf and I would like for it to go back to being Letter to Client 27May2016.pdf

Another example filename is ABCDEF~20160531-162719 and I would like for it to go back to being ABCDEF. Note that this file has no extension, unlike the example above.

I need a command that I can run at the root of the affected folders that will recursively go through and find/fix the filenames.

( I use Syncthing to sync files, and restored deleted files by copying them from the .stversions directory back to where they were, but found that Syncthing appends that date/time stamp…)

Best Answer

Meet the Perl rename tool:

$ rename -n -v  's/~[^.]+//' *~*
rename(ABCDEF~20160531-162719, ABCDEF)
rename(Letter to Client 27May2016~20160531-162719.pdf, Letter to Client 27May2016.pdf)

(online man page, also see this Q)

That regex says to match a tilde, as many characters that are not dots, but at least one; and to replace whatever matched with an empty string. Remove the -n to actually do the replace. We could change the pattern to ~[-0-9]+ to just replace digits and dashes.

Sorry, you said "recursively", so lets use find:

$ find -type f -name "*~*" -execdir  rename -n -v  's/~[-0-9]+//' {} +
rename(./ABCDEF~20160531-162719, ./ABCDEF)
rename(./Letter to Client 27May2016~20160531-162719.pdf, ./Letter to Client 27May2016.pdf)

Or just with Bash or ksh, though directories with ~ followed by digits will break this:

$ shopt -s extglob       # not needed in ksh (as far as I can tell)
$ shopt -s globstar      # 'set -o globstar' in ksh
$ for f in **/*~* ; do 
    g=${f//~+([-0-9])/}; 
    echo mv -- "$f" "$g" 
  done
mv -- ABCDEF~20160531-162719 ABCDEF
mv -- Letter to Client 27May2016~20160531-162719.pdf Letter to Client 27May2016.pdf

Again, remove the echo to actually do the rename.

Related Solutions

Command Line Rename – How to Remove URI Encoding from File Names

On Debian and derivatives (including Ubuntu), you could use rename, which applies a Perl expression to each file name.

rename 's/%20/ /g' L*
        |  |  | |   |
        |  |  | |   +--- Files to match
        |  |  | +------- globally
        |  |  +--------- with space
        |  +------------ %20
        +--------------- Substitute

I would consider using underscore instead of space – as it generally makes life easier in the cli world.

To generalise to all URI encoding:

rename 'use URI::Escape; $_ = uri_unescape $_' *%*

Faster way to rename duplicate files (identified by fdupes) in another directory

In my experience fdupes can be inconsistent in the order that it outputs files (I have had my own problems using the --delete option). This should be fairly robust as it doesn't require the files to be in a specific order (as as long as there are always two dupes in different folders):

# note no trailing slash
source_dir=/home/articles
target_dir=/external/articles

fdupes "$target_dir" "$source_dir" |
  while IFS= read file; do
    case "$file" in
      "$source_dir/"*)
         source=${file##*/}
         ;;
      "$target_dir/"*)
         target=$file
         ;;
      '')
         if [ "$source" ] && [ "$target" ]; then
           echo mv -i "$target" "$target_dir/$source"
         fi
         unset source target
         ;;
    esac
  done

This will just print out the mv commands, remove the echo when you are sure you have what you want. Also the -i option for mv will prompt you if it is going to overwrite anything.

Best Answer

Related Solutions

Command Line Rename – How to Remove URI Encoding from File Names

Faster way to rename duplicate files (identified by fdupes) in another directory

Related Question