Mac – Can the files being transferred affect/slow the data throughput

ethernetperformancetime-capsuletime-machine

I'm currently trying to carry out the initial backup of an iMac using Time Machine, backing up to a Time Capsule (3TB, the older 'flat' type). The TC is connected via Ethernet and for reasons to do with our home network I've given it a static IP address as per this article: https://www.macworld.com/article/3170186/network-storage/how-to-configure-a-time-capsule-as-an-ethernet-only-storage-device.html

Wifi on the iMac is turned off, so the connection is definitely via Ethernet. Nothing else is connected to either the iMac or the TC (wireless is off on the TC too) other than a printer connected via USB to the TC.

The data on the iMac totals about 500GB ad for the first 300GB of the backup everything proceeded as expected with data-sent rates (observed via Activity Monitor's Network tab) of 20-35 MB/sec — about what I expect to see from Ethernet. Around the 300BG mark, however, data rates fell through the floor to about 5KB/sec, about one 5000th of what I should be getting.

I left it to continue overnight but in the morning the data rates were the same and in 6 hours it had transferred only a fraction of 1GB to the backup, so I quit the TM backup and started it again. The backup process seems to have picked up where it left off, but the data rates are still crummy — typically giving me 1.3MB/sec with occasion spikes up to 20MB/sec and frequent drops back down into the KB range.

Since I know that this ethernet setup can and has supported decent speeds (even earlier on in the same process), presumably there is nothing inherently wrong with the network itself. I'm puzzled as to what might be causing the slowdown. I wondered if it might be caused bu the backup processing certain types of files (very large monolithic ones, or huge numbers of small files, for example)?

Is that feasible?

[Update: the throughput has currently jumped back up to 20-30MB/sec. Nevertheless, my question still stands: can the particular files being backed up affect the data rate?]

Best Answer

Yes, the nature of the specific files can affect the data rate.

Especially if you have lots of small files the transfer rate is usually a lot worse than if you have few large files.

The reason for that being that the backup software has to use time to open/close each and every small file and possible transfer data at small block sizes (i.e. each file could have only few bytes in them).

Compare that to large files where the number of file system calls is low, and it is possible to transfer data at the optimal block sizes from disk and further on (might for example read from disk in 4 MB blocks or similar).