icloudmacos
Uploading of Desktop and Documents to iCloud hasn't worked for me for months (macOS 10.12).
I have now started from scratch with a brand-new install, and am trying to sync just ~70MB of files that are sitting on the desktop.
It's been stuck on "77.8MB of 77.8MB" for 3 days.
Why does this happen, and how can I fix it?
I've looked at possible solutions (like logging out etc.), but this is a fresh install, shouldn't it just work?
This is not a complete answer, in that I am not sure if there is a method to force or otherwise unthrottle the uploads. However, it may be helpful to some to be able to better understand what is happening during the uploads. I believe there are some cases where the Photos system will change some parameters of how it is uploading, but can't say for certain the reasons or ways of affecting it (hopefully someone can edit this in the future if they garner more information!).
As of Sierra (macOS 10.12), the way to see the logs for the iCloud Photo Library system is as follows (you can likely achieve something similar in the Console app too):
log show --last 1h --style syslog --predicate '(subsystem beginswith[cd] "com.apple.photos.cpl")'
log stream --style syslog --predicate '(subsystem beginswith[cd] "com.apple.photos.cpl")'
The first command above will show the last 1hour of logs, the second will stream new logs from the time it is run. You may get output along the lines of the following (this is style=compact, syslog may be easier to read though)
2017-12-18 14:02:42.541 Df cloudphotosd[373:2205bf] [com.apple.photos.cpl:engine.scheduler] Client has put some changes in push repository
2017-12-18 14:02:42.541 Df cloudphotosd[373:2205bf] [com.apple.photos.cpl:engine.scheduler] We are asked to schedule a sync session from CPLEngineSyncManagerStatePush
2017-12-18 14:02:52.924 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] Notified that Server has changes to be pulled
2017-12-18 14:02:52.924 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] We are in the middle of CPLEngineSyncManagerStatePush and requested CPLEngineSyncManagerStatePull will happen later in the sync session.
2017-12-18 14:02:52.925 Df cloudphotosd[373:221837] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 72.56s (48 resources uploaded): <private>
2017-12-18 14:02:52.941 Df cloudphotosd[373:221a88] [com.apple.photos.cpl:engine.scheduler] Notified that Server has changes to be pulled
2017-12-18 14:02:52.941 Df cloudphotosd[373:221a88] [com.apple.photos.cpl:engine.scheduler] We are in the middle of CPLEngineSyncManagerStatePush and requested CPLEngineSyncManagerStatePull will happen later in the sync session.
2017-12-18 14:02:53.012 Df cloudphotosd[373:221837] [com.apple.photos.cpl:engine.scheduler] Informing transport of significant work ending
2017-12-18 14:02:53.012 Df cloudphotosd[373:221837] [com.apple.photos.cpl:engine.transport.cloudkit] Setting significant work not pending, had been set for 11.69s
2017-12-18 14:04:01.865 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] Notified that Server has changes to be pulled
2017-12-18 14:04:01.866 Df cloudphotosd[373:221956] [com.apple.photos.cpl:engine.scheduler] We are in the middle of CPLEngineSyncManagerStatePush and requested CPLEngineSyncManagerStatePull will happen later in the sync session.
These log commands gather all of the com.apple.photos.cpl (Cloud Photo Library) logs; you might want to further limit to a particular 'category', e.g. just the sync engine:
log show --last 1h --style compact --predicate '(subsystem beginswith[cd] "com.apple.photos.cpl") && (category beginswith[cd] "engine.sync")'
Which will show just the bulk transport messages:
2017-12-18 13:13:31.427 Df cloudphotosd[373:20ff1e] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 37.62s (40 resources uploaded): <private>
2017-12-18 13:15:12.093 Df cloudphotosd[373:21009e] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 100.20s (40 resources uploaded): <private>
2017-12-18 13:18:50.131 Df cloudphotosd[373:210ec1] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 152.81s (44 resources uploaded): <private>
2017-12-18 13:19:22.753 Df cloudphotosd[373:2126dc] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 32.26s (44 resources uploaded): <private>
2017-12-18 13:19:56.137 Df cloudphotosd[373:212866] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 33.00s (60 resources uploaded): <private>
2017-12-18 13:20:32.789 Df cloudphotosd[373:212a7c] [com.apple.photos.cpl:engine.sync.pushtotransport] Transport successfully uploaded in 36.22s (44 resources uploaded): <private>
These show how many resources (photos or videos?) are uploaded in each 'chunk', and you can see how much time has elapsed between them. I've yet to find a particular method behind how it chooses the size of the chunk, or the delay between them, but they seem to have varied for me at different times of the day, and whether I'm using the computer/Photos or not (though this could easily be due to other bandwidth usage). I suspect the number of resources is indeed photos/videos, because there are also many engine.sync.pushtotransport events with 0 resources that take less than a second, which I suspect are syncing faces, albums, etc (at least, since High Sierra, now that faces are synced to iCloud Photo Library).
They may be selected with a minimum chunk size, or based on size of recent successfully uploaded 'chunks'. My guess is they're selected to take approximately 30 seconds (based on recent chunk upload speeds), since most of mine are at least 30 seconds (and the distribution of times looks vaguely Poisson, though I've not plotted it to test). One particular thing I've noticed in the logs is messages about the Photos app itself being opened/closed (or foregrounded/backgrounded):
2017-12-18 14:33:13.367 Df cloudphotosd[373:22a51f] [com.apple.photos.cpl:engine.foreground] Client is in foreground
2017-12-18 14:33:13.367 Df cloudphotosd[373:22a51f] [com.apple.photos.cpl:engine.scheduler] Some client is in foreground, current required state is CPLEngineSyncManagerStatePush
2017-12-18 14:33:18.517 Df cloudphotosd[373:2299dd] [com.apple.photos.cpl:engine.foreground] Client is in background
The fact that this is logged suggests to me that the sync engine considers this in its algorithm for synchronising photos, e.g. it may reduce/throttle the sync activity when the Photos app is open or in the foreground (probably to ensure a better user experience). This is certainly the case for face/scene analysis (they are stopped/throttled while Photos is in the foreground/opened), so it might be a little faster to quit Photos. That said, the messages aren't particularly enlightening, and it still won't max out my upload bandwidth when I do this.
Sorry that this doesn't answer the question in a completely useful way, but it at least might help understand the situations in which uploads are completed faster or slower, even if Apple don't give commands or settings to 'force' or de-throttle the uploads.
More speculatively, I'm wondering if there's any undocumented sysctl settings like the one for Time Machine to enable/disable throttling of particular iCloud Library functions.
Apple doesn't list a required/minimum free space, so you'd need to work with Apple support to get that sorted. As for the cache, you've already done all you can.
iOS has several low storage warnings it sends to each and every app to clean up cached files and with < 500 MB of free space, I'd wager you've exercised those limits continuously and it's surprising to me if this is the only thing that is broken on your device right now.
I would:
Also, Apple does offer debugging profiles for iOS cloud sync if you call in for support and if their "cookbook" method doesn't yield results. If your case get escalated to engineering specific error messages contained in the debug logs can often assist you in further diagnosing why all the photos are not syncing.
You could also just push the photos to a computer or other iOS device using AirDrop or USB and upload them to the cloud from another device and then remove iCloud Photos temporarily from your iPhone to allow enough space to resume normal operations and then re-enable the library with the option to reduce file size.
Basically, the system is designed to stop working when there isn't enough space for temporary files to work. This is the same design as OS X and other unix in general.
These operating systems use free storage space to cache things and make your experience better by accelerating slow resources like network and CPU by saving work when it's done to reuse it later. When you deprive the system of that buffer, things slow and eventually stop.
Best Answer
All programs are supposed to "just work", however bugs are unavoidable when humans are programming.
With a brand new install it seems that logging out or rebooting the Mac to test if this fixes the problem shouldn't be a problem. So you might want to give that a try.
If you want to fix the problem without logging out or rebooting, try opening a terminal and running:
This should give you a reasonably readable output of log info from the syncing process. Hopefully this will give you a clue as to why the process isn't progressing.
You can also try simply killing the bird process that handles the synchronization. When it starts again it might progress to completion.