(I'm using xcp-ng/xen as my hypervisor but general question as I also wonder this about qemu/kvm and virtualbox when I use them)
whenever I make a VM, I have an option to give it as many cores as I want, up to however many the system has anyway…and I can give all my VMs all my cpu cores if I want, which seems to imply they will share them.
which then brings up the question in my head: is there a reason not to give all my VMs all my cores, if the host system is just doing hypervisor duty?
Best Answer
Because you will increase the amount of context switching that happens when any of the VMs are under any non-negligible load. Context switching is much more expensive when running under a hypervisor. Here are some recent performance measurements of virtualization overhead I made.
There are other benefits to limiting each VM to fewer CPU cores that you have - for example, it means that one VM, if it goes rogue, cannot make everything else on the machine grind to a halt. There are also potential security concerns with VMs sharing CPUs, but that is not too important of you trust all the VMs you are running (e.g. spectre / meltdown style attacks).