Troubleshooting is a process of elimination and often requires patience, but since it seems you don't have a way to bring on a freeze - and because of the time in between incidents of this happening - it's going to be somewhat slower to diagnose.

While I understand your instinct that it could be a bug with WindowServer, it's actually much more likely to be an issue with how well one of your installed apps is communicating with WindowServer. I'm not familiar with all of the apps you have installed, but if you have anything that hasn't been updated in a long time it may be worth isolating that for a while (i.e. not using it) to see if that makes a difference.

One thing worth doing regardless is to boot your iMac into Safe Mode.

Boot into Safe Mode

Follow these steps to boot your iMac into Safe Mode:

1. Fully shut down your iMac
2. Restart your iMac
3. Immediately press the Shift key and keep it down
4. Let go of the Shift key when you see the login window (NOTE: If you have FileVault enabled you may need to log in twice).
5. Now use your iMac in Safe Mode until it crashes again or until you're satisfied it's not going to crash (i.e. you may be at this step for some days). Of course, some things don't work in Safe Mode, so you may need to exit Safe Mode earlier and boot normally anyway.
6. Exit Safe Mode by restarting your iMac as normal
7. Once again, test to see if your iMac still crashes

Let me know how you go, although it may take you quite some time to know if doing this has helped.

NOTE: Even just booting into Safe Mode temporarily can be enough to resolve some issues, so it's still worth doing even if you can't stay in Safe Mode for too long.

TL;DR

I found out that my problem was caused by the no longer maintained MagicPrefs software. Uninstalled it, problem gone!

How I found the culprit

Today the problem occurred again, and I think I managed to find the culprit! Once the problem occurred, I opened Activity Monitor to see if I could spot some process doing weird. The first thing I noticed, was that the CPU load was low overall. No processes were taking up more than 20% CPU.

Then I right clicked on the column headers to add some more columns, hoping I would find a clue. I added the column showing if a process was unexpectedly stopped. (My MacOS is set to Dutch, the column is called Plots. beëind. in Dutch, I don't know the exact column name in English.) I immediately noticed that the hidd process had a "Yes" in that column.

So I went Googling and found this explanation of what the hidd process is. It's the Human Interface Device Daemon. The article states:

If high resource usage persists, the likely culprit is third party software. If you’ve recently installed drivers for a third party input device, or software that lets you do things like customize your key bindings, this could hypothetically be the issue. Try uninstalling this software, then see if that solves the issue.

That made me think about software I have installed that has to do with Human Interface Devices. There wasn't anything installed recently, but I was using MagicPrefs for some years. I went to their website to see if there were any updates and saw this notice:

MagicPrefs is more than 5 years old now and is no longer mantained, the functionality it provided was made possible by Api's that are now deprecated and reverse engineered private Apple frameworks. Apple is not likely to replace the deprecated Api's nor provide access to the underlying multitouch technology so that's the end of the line for MagicPrefs and similar software.

TLDR if it doesn't work anymore thats it, if it still works then its just a matter of time until it won't.

When I uninstalled MagicPrefs, the freezing problem immediately disappeared! I think that proofs that MagicPrefs was the culprit. A pity I can't middle click any more, but I'm relieved that this weird problem is gone now!