I'm working on an archival project and currently I'm storing 514600 images (ranging from 200kb to 2mb) on my 4TB NAS drive. The drive itself is formatted as ext4 and I'm writing the files over the network via cifs/smb.
Is there any implications of storing such a huge number of files? Will the filesystem be victim to fragmentation?
Best Answer
Fragmentation ocures when a file already on disk is changed such that it becomes larger than it previously was, and no file smaller than your cluster size should end up fragmented. to correctly answer your question, we need to know your cluster size, the percentage of the drive in use, and whether the files get changed after the initial allocation of disk space.
IO operations on many small files vs a few large files are much much slower, due to all the files system overhead (and scan/seek time), so if you consider that degraded performance, then yes. it really depends more on whether you are accessing 5000 of them at once, or just a few at a time.
Ext4 in particular doesn't require defragmentation the same way windows file systems do because of the way it allocates space to files, so as long as your partition is not 90% full, I wouldn't worry about that.