MacOS – How to ZFS be used with encryption on OS X

encryptionmacoszfs

Ideals:

I'll begin with an answer should be good for released and beta builds of ZEVO on OS X 10.7.4. It does not meet the ideals above, but it's food for thought.

Wish for Mountain Lion and greater versions of OS X to offer better support in these areas.

Answers may be added or improved after any of the following:

  • release of OS X 10.8
  • improvements to ZEVO
  • improvements to ZFS (encrypting ZFS is currently limited to Oracle Solaris).

Best Answer

OS X 10.7.4 Core Storage for encryption, ZEVO for ZFS

Example

Objective: one physical disk, five volumes. One of five non-encrypted, one encrypted with a passphrase, three encrypted with a different passphrase.

Starting point:

  • /dev/disk2 – a physical disk
  • /dev/disk2s2 – a JHFS+ volume to be kept, unprotected
  • /dev/disk2s3 – a slice to be destroyed, to be reused with encryption and ZFS.

Hints

Since HFS+ has a single global lock for all metadata updates (held across the I/O) it is highly sensitive to disk latencies. So whilst the mixture of ZFS with JHFS+ on a single disk is possible, for most use cases the file system mixture is not recommended.

This example also makes some use of compression, and of child file systems.

Summary of commands

diskutil coreStorage create pusscats /dev/disk2s3
diskutil coreStorage createVolume 370260EF-D772-4739-BAF6-F4CD6F49A60D jhfs+ Billski 50% -passphrase
diskutil coreStorage createVolume 370260EF-D772-4739-BAF6-F4CD6F49A60D jhfs+ Pickle 100% -passphrase
diskutil unmount disk3 && diskutil unmount disk4
sudo zpool create -O casesensitivity=insensitive -O normalization=formD Billski /dev/disk3
sudo zpool create -O casesensitivity=insensitive -O normalization=formD -O compression=on Pickle /dev/disk4
zfs create Pickle/button
zfs create Pickle/button/nose
sudo chown gjp22 /Volumes/Billski
sudo chown gjp22 /Volumes/Pickle
zpool export Billski && zpool export Pickle
diskutil unmount /Volumes/slice\ two

The output, in Terminal

sh-3.2$ diskutil list disk2
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *8.0 GB     disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                  Apple_HFS slice two               1.5 GB     disk2s2
   3:                  Apple_HFS slice three             6.0 GB     disk2s3
sh-3.2$ diskutil coreStorage create pusscats /dev/disk2s3
Started CoreStorage operation
Unmounting disk2s3
Touching partition type on disk2s3
Adding disk2s3 to Logical Volume Group
Creating Core Storage Logical Volume Group
Switching disk2s3 to Core Storage
Waiting for Logical Volume Group to appear
Discovered new Logical Volume Group "370260EF-D772-4739-BAF6-F4CD6F49A60D"
Core Storage LVG UUID: 370260EF-D772-4739-BAF6-F4CD6F49A60D
Finished CoreStorage operation
sh-3.2$ diskutil coreStorage createVolume 370260EF-D772-4739-BAF6-F4CD6F49A60D jhfs+ Billski 50% -passphrase
Passphrase for new volume:
Confirm new passphrase:
Started CoreStorage operation
Waiting for Logical Volume to appear
Formatting file system for Logical Volume
Initialized /dev/rdisk3 as a 3 GB HFS Plus volume with a 8192k journal
Mounting disk
Core Storage LV UUID: 22CF33F0-9EB1-424C-AFFA-FFA45A75A828
Core Storage disk: disk3
Finished CoreStorage operation
sh-3.2$ diskutil coreStorage createVolume 370260EF-D772-4739-BAF6-F4CD6F49A60D jhfs+ Pickle 100% -passphrase
Passphrase for new volume:
Confirm new passphrase:
Started CoreStorage operation
Waiting for Logical Volume to appear
Formatting file system for Logical Volume
Initialized /dev/rdisk4 as a 3 GB HFS Plus volume with a 8192k journal
Mounting disk
Core Storage LV UUID: 500E7D8F-27A3-4704-BF9F-5AE5D6BB6982
Core Storage disk: disk4
Finished CoreStorage operation
sh-3.2$ diskutil unmount disk3 && diskutil unmount disk4
Volume Billski on disk3 unmounted
Volume Pickle on disk4 unmounted
sh-3.2$ sudo zpool create -O casesensitivity=insensitive -O normalization=formD Billski /dev/disk3
Password:
sh-3.2$ sudo zpool create -O casesensitivity=insensitive -O normalization=formD -O compression=on Pickle /dev/disk4
sh-3.2$ zfs create Pickle/button
cannot mount '/Volumes/Pickle/button': failed to create mountpoint
filesystem successfully created, but not mounted
sh-3.2$ zfs create Pickle/button/nose
cannot mount '/Volumes/Pickle/button/nose': failed to create mountpoint
filesystem successfully created, but not mounted
sh-3.2$ sudo chown gjp22 /Volumes/Billski
sh-3.2$ sudo chown gjp22 /Volumes/Pickle
sh-3.2$ zpool export Billski && zpool export Pickle
sh-3.2$ diskutil unmount /Volumes/slice\ two
Volume slice two on disk2s2 unmounted
sh-3.2$ clear

At this point, with all volumes of the physical disk unmounted, you can disconnect the disk. Reconnection should allow use of the child file systems: 

sh-3.2$ zfs list
NAME                  USED   AVAIL   REFER  MOUNTPOINT
Billski             1.04Mi  2.37Gi   684Ki  /Volumes/Billski
Pickle              1.87Mi  2.37Gi   472Ki  /Volumes/Pickle
Pickle/button        808Ki  2.37Gi   412Ki  /Volumes/Pickle/button
Pickle/button/nose   396Ki  2.37Gi   396Ki  /Volumes/Pickle/button/nose
sh-3.2$ clear





sh-3.2$ mount | grep /dev/
/dev/disk1 on / (hfs, local, journaled)
/dev/disk0s2 on /Volumes/11E53 (hfs, local, journaled)
/dev/disk2s2 on /Volumes/slice two (hfs, local, nodev, nosuid, journaled, noowners)
/dev/disk4 on /Volumes/Billski (zfs, local, journaled)
/dev/disk6 on /Volumes/Pickle (zfs, local, journaled)
/dev/disk6s1 on /Volumes/Pickle/button (zfs, local, journaled)
/dev/disk6s1s1 on /Volumes/Pickle/button/nose (zfs, local, journaled)
sh-3.2$ clear





sh-3.2$ ls -l /Volumes
total 40
drwxr-xr-x@ 32 root   wheel  1156  3 Jun 09:08 11E53
lrwxr-xr-x   1 root   admin     1  4 Jun 15:17 11E53-2012-06-03 -> /
drwxr-xr-x   5 root   wheel     5  4 Jun 22:27 Billski
drwxr-xr-x   6 root   wheel     6  4 Jun 22:33 Pickle
drwxrwxr-x   7 gjp22  staff   306  4 Jun 21:41 slice two
sh-3.2$ clear





sh-3.2$ diskutil list
/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *160.0 GB   disk0
   1:                        EFI                         209.7 MB   disk0s1
   2:                  Apple_HFS 11E53                   25.0 GB    disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
   4:          Apple_CoreStorage                         25.0 GB    disk0s4
   5:                 Apple_Boot Recovery HD             650.0 MB   disk0s5
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS 11E53-2012-06-03       *24.7 GB    disk1
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *8.0 GB     disk2
   1:                        EFI                         209.7 MB   disk2s1
   2:                  Apple_HFS slice two               1.5 GB     disk2s2
   3:          Apple_CoreStorage                         6.0 GB     disk2s3
   4:                 Apple_Boot Boot OS X               134.2 MB   disk2s4
/dev/disk3
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.9 GB     disk3
   1:                        EFI                         209.7 MB   disk3s1
   2:                        ZFS                         2.6 GB     disk3s2
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:             zfs_pool_proxy Billski                *2.6 GB     disk4
/dev/disk5
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.9 GB     disk5
   1:                        EFI                         209.7 MB   disk5s1
   2:                        ZFS                         2.6 GB     disk5s2
/dev/disk6
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:             zfs_pool_proxy Pickle                 *2.6 GB     disk6
   1:       zfs_filesystem_proxy button                  2.5 GB     disk6s1
   2:       zfs_filesystem_proxy nose                    2.5 GB     disk6s1s1
sh-3.2$ clear





sh-3.2$ diskutil coreStorage list
CoreStorage logical volume groups (2 found)
|
+-- Logical Volume Group ED97CA61-2077-45CE-9CBB-8AA0A7600CB5
|   =========================================================
|   Name:         b
|   Sequence:     1
|   Free Space:   0 B (0 B)
|   |
|   +-< Physical Volume 5C79B2B6-CE01-4F02-9222-2AD898791AC5
|   |   ----------------------------------------------------
|   |   Index:    0
|   |   Disk:     disk0s4
|   |   Status:   Online
|   |   Size:     24999997440 B (25.0 GB)
|   |
|   +-> Logical Volume Family 7A9C5B98-287F-4F6B-9F9C-9641EFBC16E1
|       ----------------------------------------------------------
|       Sequence:               11
|       Encryption Status:      Unlocked
|       Encryption Type:        AES-XTS
|       Encryption Context:     Present
|       Conversion Status:      Complete
|       Has Encrypted Extents:  Yes
|       Conversion Direction:   -none-
|       |
|       +-> Logical Volume 3A130C9E-63AE-4F00-A1C3-E750B1570381
|           ---------------------------------------------------
|           Disk:               disk1
|           Status:             Online
|           Sequence:           5
|           Size (Total):       24681226240 B (24.7 GB)
|           Size (Converted):   -none-
|           Revertible:         Yes (unlock and decryption required)
|           LV Name:            11E53-2012-06-03
|           Volume Name:        11E53-2012-06-03
|           Content Hint:       Apple_HFS
|
+-- Logical Volume Group 370260EF-D772-4739-BAF6-F4CD6F49A60D
    =========================================================
    Name:         pusscats
    Sequence:     1
    Free Space:   0 B (0 B)
    |
    +-< Physical Volume F64F6909-25BF-4638-8647-CD058D60968B
    |   ----------------------------------------------------
    |   Index:    0
    |   Disk:     disk2s3
    |   Status:   Online
    |   Size:     6020829184 B (6.0 GB)
    |
    +-> Logical Volume Family 4D42970B-FAB7-4EFE-B120-2057A9A7A7E5
    |   ----------------------------------------------------------
    |   Sequence:               2
    |   Encryption Status:      Unlocked
    |   Encryption Type:        AES-XTS
    |   Encryption Context:     Present
    |   Conversion Status:      NoConversion
    |   Has Encrypted Extents:  Yes
    |   Conversion Direction:   -none-
    |   |
    |   +-> Logical Volume 22CF33F0-9EB1-424C-AFFA-FFA45A75A828
    |       ---------------------------------------------------
    |       Disk:               disk3
    |       Status:             Online
    |       Sequence:           2
    |       Size (Total):       2934915072 B (2.9 GB)
    |       Size (Converted):   -none-
    |       Revertible:         No
    |       LV Name:            Billski
    |       Content Hint:       Apple_HFS
    |
    +-> Logical Volume Family 3A009EF4-4B3D-4BC8-8B47-EFE88B2BA12A
        ----------------------------------------------------------
        Sequence:               2
        Encryption Status:      Unlocked
        Encryption Type:        AES-XTS
        Encryption Context:     Present
        Conversion Status:      NoConversion
        Has Encrypted Extents:  Yes
        Conversion Direction:   -none-
        |
        +-> Logical Volume 500E7D8F-27A3-4704-BF9F-5AE5D6BB6982
            ---------------------------------------------------
            Disk:               disk5
            Status:             Online
            Sequence:           2
            Size (Total):       2934910976 B (2.9 GB)
            Size (Converted):   -none-
            Revertible:         No
            LV Name:            Pickle
            Content Hint:       Apple_HFS
sh-3.2$ clear





sh-3.2$ zpool status
  pool: Billski
 state: ONLINE
 scan: none requested
config:

    NAME                                         STATE     READ WRITE CKSUM
    Billski                                      ONLINE       0     0     0
      GPTE_7CBD336C-568A-41AE-8E61-291CBD8EF783  ONLINE       0     0     0  at disk3s2

errors: No known data errors

  pool: Pickle
 state: ONLINE
 scan: none requested
config:

    NAME                                         STATE     READ WRITE CKSUM
    Pickle                                       ONLINE       0     0     0
      GPTE_25BF8159-FB8D-4589-B52A-184F6675176E  ONLINE       0     0     0  at disk5s2

errors: No known data errors
sh-3.2$

The end result, in Finder

A ZFS hierarchy in Finder

Finder recognition of multiple volumes, at eject time

Known issues

In CoreStorage world the content hint Apple_HFS will be not true for the ZFS end result. I have asked Apple to allow a more suitable hint.

At unlock time, the system may appear to reject the passphrase (maybe because of the content hint mismatch). When rejection appears:

  1. wait
  2. the unlocked volume (and any child file systems) should mount
  3. escape from the password dialogue.

If you wish, use Keychain Access to add to a keychain the passphrase for the UUID of a Core Storage logical volume.

Mountain Lion no longer supports diskutil eraseDisk placement of partition maps on Core Storage logical volumes –

gpes3e-gjp4:~ gjp22$ diskutil list /dev/disk4 && diskutil list /dev/disk5
/dev/disk4
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *8.0 GB     disk4
   1:                        EFI                         209.7 MB   disk4s1
   2:          Apple_CoreStorage test                    7.7 GB     disk4s2
   3:                 Apple_Boot Boot OS X               134.2 MB   disk4s3
/dev/disk5
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS test                   *7.3 GB     disk5
gpes3e-gjp4:~ gjp22$ diskutil unmount /dev/disk5
Volume test on disk5 unmounted
gpes3e-gjp4:~ gjp22$ sudo zpool create -O casesensitivity=insensitive -O normalization=formD Billski /dev/disk5

Unable to begin erase operation: Placing a partition map on a Core Storage logical volume is not supported (-69671)
diskutil eraseDisk failed, exit status 1
gpes3e-gjp4:~ gjp22$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.8.5
BuildVersion:   12F37
gpes3e-gjp4:~ gjp22$

– thanks to mgrosman for the alert.

Repairs

Considering Apple's design of Core Storage to work primarily with content hint Apple_HFS, and the bugs in fsck_cs:

  • I don't know whether 'forcing' alternative types (ZFS, zfs_pool_proxy, zfs_filesystem_proxy and so on) might impair the ability of fsck_cs to effect repairs in case of difficulty.

Proceed with caution.

Related Question