Background:
I am planning to set up a KVM to run Windows and Linux on the same workstation, and am undecided about how to set up the disks. It is my understanding that raw disks typically improve performance. I will be doing lots of I/O intensive work, so performance is critical.
However, the set-up I am planning is going to take a lot of work to set up, and I don't want to have to do it twice if I don't have to. In my experience with desktop virtualization with Vbox and VMware Player, I've gotten burned several times by corrupt VM files that wouldn't boot. This attracts me to a setup with robust protection against data corruption and good restoration features. This attracts me to ZFS.
Based on these benchmarks, ZFS is better for VM storage than other filesystems, but it does not compare with raw disk passthrough.
http://www.ilsistemista.net/index.php/virtualization/47-zfs-btrfs-xfs-ext4-and-lvm-with-kvm-a-storage-performance-comparison.html
My Questions:
-
How does the speed of VM files run from a ZFS pool compare with raw disk mode, in particular with Winodws/NTFS as the guest?
-
If you were building a dual-OS workstation, how would you weigh the merits of these two set-ups?
-
Is there anything important with respect to this that I don't seem to be considering?
Best Answer
ZFS can be (much) faster or safer in the following situations:
If you are contemplating using some other software RAID or bios RAID solution (e.g., not a dedicated piece of hardware with a relatively powerful RAID processor on-board, significant DRAM cache, etc.) -- ZFS has the most performant software RAID
If your particular workload benefits significantly more from the Adaptive Replacement Cache algorithm that ZFS uses compared to the Least Recently Used page cache algorithm that other filesystems (and raw disk blocks) use. There have been discussions about using ARC or a similar algorithm instead of LRU in the Linux kernel, but I'm not sure anything ever came of it.
If you want to use ZFS's unique RAIDz instead of software RAID-5 to solve the write hole problem.
If you want to set up a tiered storage solution where you have faster drives (usually, SSDs) sitting in front of your slower high-capacity HDDs. ZFS is best for this because you get L2ARC and/or ZIL, which boils down to a layer of read and write caching that gives you performance that feels "SSD-like" while benefiting from total available storage capacity of your larger HDDs. Example: 2 x 6 TB HDDs in RAID-1, with 2 x 250 GB SSDs in front of them with their partitioning split between L2ARC and ZIL.
If the filesystem you're putting in a ZVOL or a file within ZFS does not support (efficent) block-layer lossless compression. LZ4 is fast and can be quite a big disk space savings. If your filesystem uses an inferior algorithm (slower or worse compression ratio or both), it can be advantageous to use LZ4 in the ZFS layer instead of within the internal filesystem.
Also, I'd recommend that you use a ZVOL instead of a "filesystem within a file" to store your VMs.
Either that, or directly use a dataset (this is the most efficient), if your VM is Linux and can directly use a ZFS dataset.