Mac – n optimal setting for the number of CPU cores exposed to a VM

Like most code, this code is not doing a good job of taking advantage of multiple cores. It may switch which core it's placing load on very rapidly (around 100 times per second), but it is only able to take full advantage of one core at a time.

On these CPUs, each physical core has its own L2 cache. The L3 cache is shared by all cores and is inclusive -- that is, any data that resides in any core's L2 cache also resides on the L3 cache.

While this may seem a waste of L3 space, it actually makes the L3 invaluable for accelerating inter-core memory operations. The primary purpose of the L3 cache is to act as a switchboard and staging area for the cores. For example, if one core wants to know if a region of memory might be cached by another core, it can check the L3 cache. If information was processed by one core and next needs to be processed by another core, they hand it off through the L3 cache rather than the slower off-chip memory. Beyond that, its performance impact is not that much except for unusual algorithms -- the L2 cache is big enough for small things and the L3 cache is too small for big things.

So while each core does have its own 256KB L2 cache and effectively 256KB reserved in the L3 cache, the balance is shared by all cores. Less important activity in other cores can harm the performance of a more important task that benefits from using L3 space. But for the reasons I mentioned, it's generally not a significant effect in practice and it's generally not worth worrying about beyond optimizing "bulk data" operations (such as compression and scanning) to minimize cache pollution. (For example, using non-temporal operations.)