Shell Script – Create Sub-Directories and Organize Files by Date

Answer fixed to get 2013-11-25 instead of 20131125

If your script runs with a bash compatible shell, the easiest solution is to replace

d=$(date -r "$x" +%Y-%m-%d)

with

d="${x:4:4}-${x:8:2}-${x:10:2}"

portable solution with expr:

d=$(expr substr "$x" 5 4)-$(expr substr "$x" 9 2)-$(expr substr "$x" 11 2)

If you need only 20131125 instead of 2013-11-25 as directory name, you can also

Solution with sed:

d=$(echo "$x" | sed 's/.*_\([0-9]*\)_.*/\1/')

The sed commands replaces the filename with the number between the underscores (=the date).

Solution with awk:

d=$(echo "$x" | awk -F _ '{print $2}')

Solution with cut:

d=$(echo "$x" | cut -d_ -f 2')

Assuming you have your desired files in a text file, you could do something like

while IFS= read -r file; do 
    echo mkdir -p ${file%/*}; 
    cp /source/"$file" /target/${file%/*}/${file##*/}; 
done < files.txt 

That will read each line of your list, extract the directory and the file name, create the directory and copy the file. You will need to change source and target to the actual parent directories you are using. For example, to copy /foo/a/a.txt to /bar/a/a.txt, change source to foo and target to bar.

I can't tell from your question whether you want to copy all directories and then only specific files or if you just want the directories that will contain files. The solution above will only create the necessary directories. If you want to create all of them, use

find /source -type d -exec mkdir -p {} /target

That will create the directories. Once those are there, just copy the files:

while IFS= read -r file; do 
    cp /source/"$file" /target/"$file"
done

Update

This little script will move all the files modified after September 8. It assumes the GNU versions of find and touch. Assuming you're using Linux, that's what you will have.

#!/usr/bin/env bash    

## Create a file to compare against.
tmp=$(mktemp)
touch -d "September 8" "$tmp"

## Define the source and target parent directories
source=/path/to/source
target=/path/to/target

## move to the source directory
cd "$source"

## Find the files that were modified more recently than $tmp and copy them
find ./ -type f -newer "$tmp" -printf "%h %p\0" | 
    while IFS= read -rd '' path file; do
        mkdir -p "$target"/"$path"
        cp "$file" "$target"/"$path"
    done

Strictly speaking, you don't need the tmp file. However, this way, the same script will work tomorrow. Otherwise, if you use find's -mtime, you would have to calculate the right date every day.

Another approach would be to first find the directories, create them in the target and then copy the files:

1. Create all directories

    find ./ -type d -exec mkdir -p ../bar/{} \;
   

2. Find and copy the relevant files

    find ./ -type f -newer "$tmp" -exec cp {} /path/to/target/bar/{} \;
   

3. Remove any empty directories

    find ../bar/ -type d -exec rmdir {} \;