Text Processing – Split File into N Pieces with Same Name in Different Directories

filessplittext processing

I want to split sourcefile.txt which contains 10000 lines, (increasing everyday) into 30 equal files. I have directories called prog1 to prog30 and I would like to save split the file into these directories with the same filename. For example /prog1/myfile.txt, /prog2/myfile.txt to /prog30/myfile.txt.

Here is my bash script called divide.sh runs in prog directory

#!/bin/bash
programpath=/home/mywebsite/project/a1/
array=/prog1/
totalline=$(wc -l < ./sourcefile.txt)   
divide="$(( $totalline / 30 ))"   
split --lines=$divide $./prog1/myfile.txt    
exit 1
fi

Best Answer

Sed version for fun:

lines=$(wc -l <sourcefile.txt)
perfile=$(( (lines+29)/30 ))     # see https://www.rfc-editor.org/rfc/rfc968.txt
last=0
sed -nf- sourcefile.txt <<EOD
$(while let $((last<lines)); do 
        mkdir -p prog$((last/perfile+1))
        echo $((last+1)),$((last+perfile)) w prog$((last/perfile+1))/myfile.txt
        : $((last+=perfile))
        done)
EOD
Related Question