ZFS – How Copy on Write Works for Large Files

ZFS is not in the official Linux kernel, and never will be unless Oracle relicenses the code under something compatible with the GPL.

This incompatibility is disputed. The main arguments in favor of ZFS being allowed on Linux systems revolve around the so-called "arm's length" rule. That rule applies in this case only if ZFS is provided as a separate module from the kernel, the two communicate only through published APIs, and both code bases can function independently of each other. The claim then is that neither code base's license taints the other because neither is a derived work of the other; they are independent, but cooperate. Nevertheless, even under this interpretation, it means the ZFS modules must still be shipped separately from the Linux kernel, which is how we see it being provided today by Ubuntu.

Quite separately from the CDDL vs GPL argument, NetApp claims they own patents on some technology used in ZFS. NetApp settled their lawsuit with Sun after the Oracle buyout, but that settlement doesn't protect any other Linux distributor. (Red Hat, Ubuntu, SuSE...)

As I see it, these are your alternatives:

• Use btrfs instead, as it has similar features to ZFS but doesn't have the GPL license conflict and has been in the mainline kernel for testing since 2.6.29 (released in January 2009).

  The main problem with btrfs is that it's had a long history of problems with its RAID 5/6 functionality. These problems are being worked out, but each time one of these problems surfaces, it resets the "stability clock."

  Another concern is that Red Hat have indicated that the next release of Red Hat Enterprise Linux will not include btrfs.

• One of the reasons Red Hat is taking that position on btrfs is that they have a plan to offer similar functionality using a different technology stack they are calling Stratis. Therefore, another option you have is to wait for Stratis to appear, with 1.0 scheduled for the first half of 2018, presumably to coincide with Red Hat Enterprise Linux 8.

• Use a different OS for your file server (FreeBSD, say) and use NFS to connect it to your Linux boxes

• Use ZFS on FUSE, a userspace implementation, which works neatly around the kernel licensing issue at the expense of a significant amount of performance

• Integrate ZFS on Linux after installing the OS.

  The license conflict makes distributing the combined system outside your organization legally questionable. I am not a lawyer, but my sense is that, patent issues aside, distributing ZFS on Linux is about as worrisome as distributing non-GPL binary drivers (such as those for certain video cards) with the system. If one of these bothers you, the other should, too.

• Switch to Ubuntu, which has been shipping ZFS kernel modules with the OS since 16.04. Canonical believes that it is legally safe to distribute the ZFS kernel module with the OS itself. You would have to decide whether you trust Canonical's opinion; consider also that they may not be willing to indemnify you if a legal issue comes up.

  Beware that it is not currently possible to boot from ZFS with Ubuntu without a whole lot of manual hackery.

Incidentally, btrfs is also backed by Oracle, but was started years before the Sun acquisition. I don't believe the two will ever merge, or one be deprecated in favor of the other due to the license conflict and patent issue. ZFS is too popular to go away, but there will continue to be demand for a ZFS alternative.

I think option 3 as you have described above is probably your best bet. The biggest problem with what you want is that ZFS really only handles this copy-on-write at the dataset/snapshot level.

I would strongly suggest avoiding using dedup unless you have verified that it works well with your exact environment. I have personal experience with dedup working great until one more user or VM store is moved in, and then it falls off a performance cliff and causes a lot of problems. Just because it looks like it's working great with your first ten users, your machine might fall over when you add the eleventh (or twelfth, or thirteenth, or whatever). If you want to go this route, make absolutely sure that you have a test environment that exactly mimics your production environment and that it works well in that environment.

Back to option 3, you'll need to set up a specific data set to hold each of the file system trees that you want to manage in this way. Once you've got it set up and initially populated, take your snapshots (one per dataset that will differ slightly) and promote then into clones. Never touch the original dataset again.

Yes, this solution has problems. I'm not saying it doesn't, but given the restrictions of ZFS, it's still probably the best one. I did find this reference to someone using clones effectively: http://thegreyblog.blogspot.com/2009/05/sparing-disk-space-with-zfs-clones.html

I'm not real familiar with btrfs, but if it supports the options that you want, have you considered setting up a separate server just to support these datasets, using Linux and btrfs on that server?