Ubuntu – Command to create sequentially numbered targeted directories and batch move files

bashcommand linedirectory

I'm creating a timelapse video by splitting a video into individual images, using G'MIC via the command line to average every 7 frames, then output the results to new images which will result in the frames for the timelapse (this way I get noise free video that looks great). As you can imagine I have a lot of frames – several thousand individual images in a directory, named sequentially (image1.tiff, image2.tiff, etc.). I found a script that works perfectly for moving the files – once… 

k=1; find source/ -type f | while read file; do
     [[ k++ -le 7 ]] && mv "$file" target/ 
done

What I need now is to make this script

  1. repeat itself until all the files are moved and
  2. create sequentially numbered target directories (1, 2, 3, etc.) as it goes so I end up with several hundred directories each with 7 images inside.

Just to clarify, directory 1 should contain images 1-7, directory 2 should contain images 8-14, etc.

Any help is greatly appreciated, I'm pretty much stuck with this one.

Best Answer

The (python) script below creates sub folders and organizes your files into the folders:

Additionally:

  • The script calculates the number of folders and adds leading zeros to the folder names for proper sorting (since you mentioned thousands of files, meaning hundreds of folders)
  • The number of files per folder may vary, depending on your setting in the head of the script

The script assumes:

  • All files have an extension, all files have the same extension
  • Your files are named image1.tiff, image2.tiff etc (no leading zeros).

The script

#!/usr/bin/env python3
import os
import math
import shutil
#---
directory = "/path/to/files"      # path to your files
n_perfolder = 7                   # number of files per sub folder
#--
# creating file list, extension
f_list = os.listdir(directory); ext = f_list[0].split(".")[-1]
# calculate number of folders
n_folders = math.ceil(len(f_list)/n_perfolder)
# creating folder names, including leading zeros
folders = [str(fn+1) for fn in range(n_folders)]
fl_names = [(len(max(folders, key=len))-len(fl))*"0"+fl for fl in folders]
# creating folders and allocate files 
for i in range(len(fl_names)):
    mkfolder = directory+"/"+fl_names[i]
    if not os.path.exists(mkfolder):
        os.makedirs(mkfolder)
        r = range(i*n_perfolder+1, i*n_perfolder+n_perfolder+1)
        for n in r:
            try:
                file = directory+"/"+"image"+str(n)+"."+ext
                target = mkfolder+"/"+"image"+str(n)+"."+ext
                shutil.copyfile(file, target)
            except FileNotFoundError:
                pass

How to use

Copy the script into an empty file, in the head section, set the directory to your files and the number of files per sub directory, save it as organize.py.

Run it by the command:

python3 /path/to/organize.py

Note

If you'd like to move the files instead of copying, replace the line:

shutil.copyfile(file, target)

by:

shutil.move(file, target)

(mind the indent!)

Related Question