IPhone – Old iMessages not transfered to newly setup iPhone

I just got a new iPhone and decided to start fresh and not restore it from an old backup (old one is cluttered with tons of apps I don't use anymore).

Everything is working just fine except from iMessage. On my old phone, I've got settings -> icloud -> messages enabled. When I check iPhone storage on my old device, I can see that messages takes up 1.7GB.

When I first enabled the same Messages switch on my new device I did get a progress bar saying 'Downloading Messages from iCloud'. I waited for maybe 15 minutes but it seemed that not much happened and stupid me did go to the settings -> icloud -> messages and disabled it. My thought was to then turn it back on again too see if it somehow got stucked and this maneuver would fix the problem.

At this time, while disabling it, I got a message telling me 'Messages have not been fully downloaded, tap Disable and Download Messages'. I'm not even sure what this means but I did disable it. After a while I enabled it again but at this time, nothing more happens regarding Messages.

When I enter Messages I can see that some of my old messages have been transfered to my new phone but not all. If I check storage on my new phone I can see that Messages is taking up 858K of storage so something seems to have gone wrong (compared to the Messages storage on my old phone).

I have tried to enable/disable icloud Message sync a number of times but nothing happens. It seems to have lost it's connection to the stored iCloud messages. If I login to iCloud from a browser I can see that Messages takes up a whopping 4.4GB! Not sure why the iCloud storage is so large compared to my old phone Message storage.

Anyway, my question is, how can i resume the transfering of my old iMessages to my newly setup iPhone?

I've also tried to logout, settings -> Apple-ID -> logout, and then logged back in again but no luck.