Linux – Is Xfce supposed to run slowly on Virtualbox

I'm having problems getting Xfce systems on VirtualBox to run smoothly. What I mean by slow is that moving or resizing windows looks very "laggy" (basically just windows jumping from one place to another). This seems to only happen on Xfce systems. Or more specifically, Xfwm.

— Host's spec —
MacBook Pro Retina
Mac OS X Mavericks
2.4GHz dual-core Intel Core i5
8GB 1600MHz RAM
256GB SSD
Intel Iris Graphics

— Guest's spec —
64 bit OS
10GB fixed VDI size
2GB RAM
128MB VRAM (both 12MB/128MB tested)
3D acceleration (both on/off tested)

— The tests I performed, where every OS have the same spec —
OS (WM) | Good performance [Y/n]
Debian 7.3 (Metacity) | Y
Ubuntu 13.10 (Compiz) | Y
Ubuntu 13.10 (LightDM) | Y
Linux Mint 16 Cinnamon (Mutter) | Y
Xubuntu 13.10 (Compiz) | Y
Xubuntu 13.10 (Xfwm4) | n
Linux Mint 16 Xfce (Xfwm4) | n
The CPU usage of guest from moving windows is about 10% and resizing is about 2%. The CPU usage of host is very small and can't be bothered to get a number. I found out that moving or resizing isn't really affected by the CPU usage. It's consistently that slow. I wrote a huge loop and let the CPU run at 100%. However, the movements of the windows still look roughly the same.

The results are leaning towards the conclusion that the poor performance is caused by XFCE, but this doesn't make any sense to me, as XFCE is supposed to be lightweight and all. There is a final test I performed, where I installed compiz on top of Xfwm. This actually made it smooth, which means it must be from the Xfwm. Is this similar to the problem in which VirtualBox can't run KDE properly? (I've read that somewhere, and have tried to install openSUSE and Kubuntu)

I am not interested in finding a smooth Linux OS on VirtualBox, but rather, to understand and maybe even fix what is going on with Xfce.

Have I installed guest additions?
Obviously yes, or else the rest of the OS wouldn't run smoothly as well. I can use full screen and resize windows, plus other good additions. I'm certain that I installed it correctly, as I have went through a tutorials (including VirtualBox docs) on each of the OS , and installed every OS at least twice to ensure everything is done correctly. In addition, this command: "lspci" returned: 00:02.0 VGA compatible controller: InnoTek Systemberatung GmbH VirtualBox Graphics Adapter

Have I looked around?
Yes, in fact, for about 7 hours non-stop on Google while testing other OS.

Have I checked out the settings of the guests, i.e. effects?
Yes, for every OS, I basically tested with all effects on and off, and it doesn't make the slightest difference.

Have I tried other hosts?
Unfortunately I won't have my Windows desktop with me for another week or so and I really want to know the answer soon. I guess the host could make a difference, but from my experience with running Ubuntu fully configured with compiz and other tools, both machines have negligible performance differences.

The end.
So sorry about this long post. The answer could well be very simple and short, that I don't need to type this much, but I really like to be specific as I hate people coming to me and asking obvious questions such as "Have you tried restarting?"