Ubuntu – How to add an SSD cache to an encrypted HDD for desktop/GUI use

dm-cacheencryptionlvmcachessd

SSHDs seem to be stuck at 4TB and 8GB SSD cache (one model is available with 1TB/32GB) while bigger SSDs are widely available and some solutions exist to to pair an HDD with an SSD in Linux. I'm looking for a solution that is easy to use on a desktop once it's set up. Ideally one needs to click one or two launcher icons, enter passwords for the encrypted storage and mount the drive in Nautilus without needing to remember any particular commands.

  • All storage and cache data must be stored encrypted.
  • A seprate large HDD is available for data migration to new partition schemes or what ever is required.

Related articles I read so far:

Best Answer

non-root LVM on LUKS

Introduction

Info:

  • I just started using LVM because of this feature and my intention was to use an encryted data partition with an SSD for caching.
  • I usually use ZFS on Linux (see also ARC) for data archiving and redundancy, which appears to be easier to administer and to setup. You may want to look at ZFS if you want to use more than one HDD, an advantage that LVM has over ZFS on Linux however is that it doesn't rely on a DKMS built kernel module which may not build and run correctly if you need to use mainline kernels from the kernel PPA for example. (May be I'm doing something wrong there with ZFS, but some articles on Phoronix read as if that is how it is.)
  • This is intended for workstation usage and GUI-way solutions are preferred, to possbily show what can be achived with GUIs currently.

Caveat:

  • Don't just add any SSD as a cache to your HDD and expect big performance increases. The BX200 with 480GB I currently use looks reasonable but showed slower write speeds than HDDs in benchmarks and product reviews. Also note that both devices have to write the data by default at the same time when not in writeback cache mode — which has a higher risk of data corruption — so don't be confused by doubled total transfer rates and slow writes in tools like indicator-multiload.

Overview:

To provide some more insight, here is what I set up in a VM after reading the articles in the question before trying it on actual hardware. The luks- named devices are automatically created when using the GUI to unlock the encrypted devices, LVM will detect that the devices have become available and will offer you to mount the filesystems in nautilus as usual.

lwbt@vxenial:~$ lsblk
…    
sdb                                           8:16   0    8G  0 disk
└─luks-6bc875f1-de30-4698-ba74-eea2c5d5bb87 252:0    0    8G  0 crypt 
  └─vg0-datalv_corig                        252:5    0  7,9G  0 lvm
    └─vg0-datalv                            252:1    0  7,9G  0 lvm
sdc                                           8:32   0    8G  0 disk
└─luks-e20dd038-9886-4895-b786-855ba4c31c7e 252:2    0    8G  0 crypt 
  ├─vg0-cache_cdata                         252:3    0    8G  0 lvm 
  │ └─vg0-datalv                            252:1    0  7,9G  0 lvm 
  └─vg0-cache_cmeta                         252:4    0   12M  0 lvm 
    └─vg0-datalv                            252:1    0  7,9G  0 lvm 


lwbt@vxenial:~$ ll /dev/mapper/
total 0
drwxr-xr-x  2 root root     180 Sep  6 02:54 ./
drwxr-xr-x 20 root root    4420 Sep  6 02:54 ../
crw-------  1 root root 10, 236 Sep  6 00:37 control
lrwxrwxrwx  1 root root       7 Sep  6 02:54 luks-6bc875f1-de30-4698-ba74-eea2c5d5bb87 -> ../dm-0
lrwxrwxrwx  1 root root       7 Sep  6 02:54 luks-e20dd038-9886-4895-b786-855ba4c31c7e -> ../dm-2
lrwxrwxrwx  1 root root       7 Sep  6 02:54 vg0-cache_cdata -> ../dm-3
lrwxrwxrwx  1 root root       7 Sep  6 02:54 vg0-cache_cmeta -> ../dm-4
lrwxrwxrwx  1 root root       7 Sep  6 02:54 vg0-datalv -> ../dm-1
lrwxrwxrwx  1 root root       7 Sep  6 02:54 vg0-datalv_corig -> ../dm-5

The output of lsblk might have an additional layer if you partition the block devices (SSD/HDD) instead of straight encrypting them with LUKS. Of course if you have no interest in LUKS encryption you can skip these steps in the following instructions which is fine too.

Instructions

Install the required packages:

sudo apt install lvm2 thin-provisioning-tools cryptsetup

The commands in the following section will destroy all existing data on the drives.

Encrypt each individual device:

sudo cryptsetup luksFormat ${device_name}
sudo cryptsetup luksOpen ${device_name} ${mapper_name_hdd_or_ssd}

Note: gnome-disks has an options to encrypt and mount encrypted devices, but it also creates EXT4 filesystems which would have to be deleted to continue following the instructions.

Then add the device as a physical volume to LVM, create a volume group (named vg0) and a logical volume (named datalv) on the HDD which will be formatted later to hold all the data:

pvcreate /dev/mapper/${mapper_name_hdd}
vgcreate vg0 /dev/mapper/${mapper_name_hdd}
lvcreate -l 100%pvs -n datalv vg0 /dev/mapper/${mapper_name_hdd}

Note: 100%pvs will create a volume which spans over the entire device you have chosen (e.g. the encrypted container or a partition). More details and options can be found in lvmcache and other related manpages.

Now extend the LVM configuration by adding the SSD to the volume group:

pvcreate /dev/mapper/${mapper_name_ssd}
vgextend vg0 /dev/mapper/${mapper_name_ssd}

Create a cache-pool volume named cache in vg0 on the SSD, then add the new cache pool as a cache to datalv:

lvcreate --type cache-pool -l 100%pvs -n cache vg0 /dev/mapper/${mapper_name_ssd}
lvconvert --type cache --cachepool vg0/cache vg0/datalv

Note: lvcreate will automatically choose the optimal sizes for cache_cdata and cache_cmeta for you this way.

After making changes to an LVM configuration it may be necessary to rescan pv, vg and lv to activate the LVM devices, if you want to avoid rebooting:

pvscan
vgscan
lvscan
vgchange -ay

The following command deactivates all LVM volumes when no name is provided:

vgchange -an

Finally it's time to format the logical volume and transfer data.

  1. Example for formatting using the Gparted GUI

    pkexec gparted /dev/mapper/vg0-datalv

  2. Example for formatting using command line tools:

    sudo mkfs.ext4 /dev/mapper/vg0-datalv

Now the new filesystem should show up in Nautilus.

Usage

The many LVM commands make the setup look quite complex while it may not even leverage the full potential of LVM to experienced users, but once it's setup and the computer rebooted you just need to mount the encrypted disks from the Unity launcher or from within gnome-disks, then mount the volume that shows up in Nautilus like any other filesystem or disk. No memorizing of complex commands is required. To my knowledge there is currently no GUI tool available to handle the LVM setup of a cache pool in Ubuntu.

Notes:

  • I prefer to mount the smaller encrypted cache SSD device before mounting the HDD
  • The cache can be removed and replaced see Cache Removal in lvmcache manpage.

  • To display all metrics about the cache you can use the following command:

    sudo lvs -o+cache_total_blocks,cache_used_blocks,cache_dirty_blocks,cache_read_hits,cache_read_misses,cache_write_hits,cache_write_misses,cachemode,cache_policy,cache_settings
Related Question