Ubuntu – Move files from subfolders

bashcommand line

I've been beating my head against this for a while, but I'm really not a good script writer. Apologies…

I'm running Ubuntu/gnome 18.10, and have a large set of pictures exported from my wifey's mac. The directory structure is:

year1
  (login dir name with spaces) - Month
    Image names
year2
  ...

as in:

2013
  May 4, 2013
    Image1.jpg
    Image2.jpg
  May 5, 2013
    Image 1.jpg
    Image 3.jpg
  June 22, 2013

What I would like, is:

2013
   January
     All the "january" images...
   February
     All the...

I can create the directories easily enough mkdir {January..December} suffices. But I cannot figure out how to walk the ugly directory tree (exported from Mac), move the images, and then delete the ugly directory.

Best Answer

Here is such script:

#!/bin/bash

# The destination where the new directory structure will be created
DEST="/tmp/new-order-pictures/"

MONTHS=('Jan' 'Feb' 'Mar' 'Apr' 'May' 'Jun' 'Jul' 'Aug' 'Sep' 'Oct' 'Nov' 'Dec')

# Walk through the first level directories, located in the current directory and go inside
for year in */
do
    cd "$year"
    # Walk through the months of the year
    for month in "${MONTHS[@]}"
    do
        # Walk through the second level directories
        for dir in */
        do
            # If we have coincidence between the name of the directory and the month
            # go inside, make a new destination directory; ignore character cases^^
            if [[ ${dir^^} =~ ${month^^} ]]
            then
                cd "$dir"
                dest="${DEST}${year}${month}"
                mkdir -p "$dest"
                find . -type f | while IFS= read -r item
                do
                    # Copy the files to the new destination and
                    # add the file's md5sum to its name to prevent files lose
                    filename=$(basename -- "$item")
                    extn="${filename##*.}"
                    name="${filename%.*}"
                    cp "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
                done
                cd ..
            fi
        done
    done
    cd ..
done

The script should be executed in the first level directory where your images are located. You should tweak the destination directory - DEST="/tmp/new-order-pictures/". This version of the script relies that all files are in directories that contain the name of a month in one way or another. Example of usage:

user@host:~/Pictures$ tree .
.
├── 2013
│   ├── January 17, 2013
│   │   ├── Image1.jpg
│   │   └── Image 3.jpg
│   ├── January 24, 2013
│   │   └── Image2.jpg
│   ├── January 25, 2013
│   │   └── Image 3.jpg
│   ├── June 22, 2013
│   │   └── image1.jpg
│   ├── May 4, 2013
│   │   └── Image1.jpg
│   └── May 5, 2013
│       ├── Image1.jpg
│       └── Image 2.jpg
└── 2014
    ├── January 17, 2014
    │   ├── Image1.jpg
    │   └── Image 3.jpg
    ├── January 24, 2014
    │   └── Image2.jpg
    ├── January 25, 2014
    │   └── Image 3.jpg
    └── May 5
        ├── Image1.jpg
        └── Image 2.jpg

12 directories, 14 files


user@host:~/Pictures$ order.sh 


user@host:~/Pictures$ tree /tmp/new-order-pictures/
/tmp/new-order-pictures/
├── 2013
│   ├── Jan
│   │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
│   │   ├── Image2-cbf4d36ff84e7ec24c05f8181236e6b8.jpg
│   │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
│   │   └── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
│   ├── Jun
│   │   └── image1-adb3bf995f1a25d008f758a7266d7be5.jpg
│   └── May
│       ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
│       ├── Image1-a66c5863e6986605cb2ca6d622ae72a0.jpg
│       └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg
└── 2014
    ├── Jan
    │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
    │   ├── Image2-cbf4d36ff84e7ec24c05f8181236e6b8.jpg
    │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
    │   └── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
    └── May
        ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
        └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg

7 directories, 14 files

In my case the script is named order.sh and it is located in ~/bin, thus I can use it as shell command. In the example you can see the directory structure is changed but the number of files is 14 in both structures.

Here is another version of the script that use mv instead of cp and will deal also with the files which are not in directories that contain the name of a month. Before running this script it is a good idea to create a backup copy of the original directory structure.

#!/bin/bash

# The destination where the new directory structure will be created
DEST="/tmp/new-order-pictures/"

MONTHS=('Jan' 'Feb' 'Mar' 'Apr' 'May' 'Jun' 'Jul' 'Aug' 'Sep' 'Oct' 'Nov' 'Dec')

# Walk through the first level directories, located in the current directory and go inside
for year in */
do

    cd "$year"

    # Walk through the months of the year
    for month in "${MONTHS[@]}"
    do
        # Walk through the second level directories
        for dir in */
        do
            # If we have coincidence between the name of the directory and the month
            # go inside, make a new destination directory; ignore character cases^^
            if [[ ${dir^^} =~ ${month^^} ]]
            then

                cd "$dir"
                dest="${DEST}${year}${month}"
                mkdir -p "$dest"

                while IFS= read -r item
                do
                    # Copy the files to the new destination and
                    # add the file's md5sum to its name to prevent files lose
                    filename=$(basename -- "$item")
                    extn="${filename##*.}"
                    name="${filename%.*}"
                    mv "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
                done < <(find . -type f)

                cd ..

            fi

        done

    done

    # Dial with the rest of the files for that $year

    dest="${DEST}${year}other"

    while IFS= read -r item
    do
        mkdir -p "$dest"
        filename=$(basename -- "$item")
        extn="${filename##*.}"
        name="${filename%.*}"
        mv "$item" "${dest}/${name}-$(md5sum "$item" | cut -f1 -d' ').${extn}"
    done < <(find . -type f)

    cd ..

done

Example of usage:

user@host:~/Pictures$ tree .
.
├── 2013
│   ├── January 17, 2013
│   │   ├── Image1.jpg
│   │   ├── Image 3.jpg
│   │   └── video 7.mpg
│   ├── January 25, 2013
│   │   └── Image 3.jpg
│   ├── June 22, 2013
│   │   └── image1.jpg
│   └── May 5, 2013
│       ├── Image1.jpg
│       └── Image 2.jpg
└── 2014
    ├── Apr 7
    │   ├── Image1.jpg
    │   └── Image 2.jpg
    ├── Image 2.jpg
    ├── January 11, 2014
    │   ├── Image1.jpg
    │   └── Image 3.jpg
    ├── some other name
    │   └── some other name file inside.jpg
    ├── some other name file inside.jpg
    └── video 1.avi

9 directories, 15 files

user@host:~/Pictures$ order.sh 

user@host:~/Pictures$ tree /tmp/new-order-pictures/
/tmp/new-order-pictures/
├── 2013
│   ├── Jan
│   │   ├── Image1-7b71d9fdfe5b15a2d1a4968c195f93ae.jpg
│   │   ├── Image 3-0bca5188fd3f3eb470533fdaf0630633.jpg
│   │   ├── Image 3-6a83880cae1aa57e19a7c45de7759e68.jpg
│   │   └── video 7-86764d9565469adfb22c8ef4f0b9c04f.mpg
│   ├── Jun
│   │   └── image1-adb3bf995f1a25d008f758a7266d7be5.jpg
│   └── May
│       ├── Image1-511d541b35fcb38af8ada18d7961268c.jpg
│       └── Image 2-c34ffc32ce5d3901e1ad89b9fd15a877.jpg
└── 2014
    ├── Apr
    │   ├── Image1-3c19da25e0e56ef0fc752a9e4f75b190.jpg
    │   └── Image 2-dcc35e86de393a014ac62e8c4390c7e6.jpg
    ├── Jan
    │   ├── Image1-ae34289b0bc5258f286165745ff3c258.jpg
    │   └── Image 3-1724adf2dfcc1d4a0dc50cb38ad2c510.jpg
    └── other
        ├── Image 2-eff5208f7eee6a536e48f9982b918dfb.jpg
        ├── some other name file inside-7d0a68e0b4e9cc3928744cb83f4d1136.jpg
        ├── some other name file inside-c2dd637e94a9025c3e1004d66f59539c.jpg
        └── video 1-c277d93a2427bedf3f0b8ae07427edb9.avi

8 directories, 15 files

After that you can go inside the destination directory and use the rename command within for loop to deal with the long names:

# For each directory on the second level
for dir in */*
do
    cd "$dir"
    rename 's/^.*(\.[0-9a-zA-Z]+)$/our $i; sprintf("Image-%03d$1", 1+$i++)/e' *
    cd ..
    cd ..
done

Example:

user@host:~/Pictures$ cd /tmp/new-order-pictures/

user@host:/tmp/new-order-pictures$ for dir in */*; do cd "$dir"; rename 's/^.*(\.[0-9a-zA-Z]+)$/our $i; sprintf("Image-%03d$1", 1+$i++)/e' *; cd ..; cd ..; done

user@host:/tmp/new-order-pictures$ tree .
.
├── 2013
│   ├── Jan
│   │   ├── Image-001.jpg
│   │   ├── Image-002.jpg
│   │   ├── Image-003.jpg
│   │   └── Image-004.mpg
│   ├── Jun
│   │   └── Image-001.jpg
│   └── May
│       ├── Image-001.jpg
│       └── Image-002.jpg
└── 2014
    ├── Apr
    │   ├── Image-001.jpg
    │   └── Image-002.jpg
    ├── Jan
    │   ├── Image-001.jpg
    │   └── Image-002.jpg
    └── other
        ├── Image-001.jpg
        ├── Image-002.jpg
        ├── Image-003.jpg
        └── Image-004.avi

8 directories, 15 files

Or you can change (\.[0-9a-zA-Z]+) with (\.jpg), then on next iteration with (\.mpg) (respectively Image- with Video-), etc. References about this usage of rename:

Related Question