Is a ZFS storage array portable across operating systems & CPU Architectures

filesystemsportabilityraidzfs

I know that ZFS is available on variety of operating systems Solaris, Illumos, Linux, BSD etc.

Suppose I have a bunch of disks that are part of ZFS raid file system that was created on Linux, could those drives be taken out of the Linux machine and put into a machine running BSD or Illumos and just work? Are there any OS specific things that are written to ZFS file systems that would make moving a disk between different implementations of ZFS a problem?

What about if the ZFS formatted disks were moved across CPU architectures say from a machine running SPARC to x86. Would the ZFS metadata be affected by endianess of the CPU architecture?

Best Answer

What really matters when dealing with ZFS pools and datasets portability is their versions. You can always import a pool using a version less or equal to the one your OS supports, and the same rule applies for datasets (i.e. file systems, zvols and snapshots).

So if you plan to move ZFS pools from some OSes to different ones, make sure you select the largest common version for pools, file systems and zvols you want to share. eg:

zpool create -o version=28 -O version=5 ...

Beware too that the Open Source branch of ZFS introduced "feature flags" which, when enabled, make the pools incompatible with Solaris. They set the pool version to 5000. On the other hand, feature flags were designed to allow some flexibility when moving a pool to an OS not supporting some features.

About architectures, ZFS objects are written using the native bit ordering of the platform. This is not a problem as all ZFS implementations are able to read both big endian and little endian objects. The endianness is stored along with all data structures. There is then no issue importing a pool from say x86 to SPARC and reciprocally.

Finally, you might have issues when dealing with pools built on non whole disks. The target OS must be able to understand the source disk partitioning. The worst case scenario would be if you create a pool based on plain files (which is not recommended outside testing) and the file system used cannot be mounted on the target platform.

Related Solutions

Linux: LUKS and multiple hard drives

You can use /lib/cryptsetup/scripts/decrypt_derived in your crypttab to automatically use the key from one disk for another.

The decrypt_derived script is part of Debian's cryptsetup package.

Small example to add the key from sda6crypt to sda5:

/lib/cryptsetup/scripts/decrypt_derived sda6crypt > /path/to/mykeyfile
cryptsetup luksAddKey /dev/sda5 /path/to/mykeyfile
ls -la /dev/disk/by-uuid/ | grep sda5
echo "sda5crypt UUID=<uuid> sda6crypt luks,keyscript=/lib/cryptsetup/scripts/decrypt_derived" >> /etc/crypttab
shred -u /path/to/mykeyfile # remove the keyfile

As it is nowadays very difficult to really delete a file, ensure that /path/to/mykeyfile is on a encrypted drive (sda6crypt would be in my example a good solution).

In general, you can add an additional security layer by using user space filesystem encryption e.g. via encfs.

Are binaries portable across different CPU architectures

No. Binaries must be (re)compiled for the target architecture, and Linux offers nothing like fat binaries out of the box. The reason is because the code is compiled to machine code for a specific architecture, and machine code is very different between most processor families (ARM and x86 for instance are very different).

EDIT: it is worth noting that some architectures offer levels of backwards compatibility (and even rarer, compatibility with other architectures); on 64-bit CPU's, it's common to have backwards compatibility to 32-bit editions (but remember: your dependent libraries must also be 32-bit, including your C standard library, unless you statically link). Also worth mentioning is Itanium, where it was possible to run x86 code (32-bit only), albeit very slowly; the poor execution speed of x86 code was at least part of the reason it wasn't very successful in the market.

Bear in mind that you still cannot use binaries compiled with newer instructions on older CPU's, even in compatibility modes (for example, you cannot use AVX in a 32-bit binary on Nehalem x86 processors; the CPU just doesn't support it.

Note that kernel modules must be compiled for the relevant architecture; in addition, 32-bit kernel modules will not work on 64-bit kernels or vice versa.

For information on cross-compiling binaries (so you don't have to have a toolchain on the target ARM device), see grochmal's comprehensive answer below.

Best Answer

Related Solutions

Linux: LUKS and multiple hard drives

Are binaries portable across different CPU architectures

Related Question