How to configure AppArmor so that KVM can start guest that has a backing file chain

apparmorkvmqemusnapshot

I'm playing around with using Qemu external snapshots for KVM (because they are so fast!) on Ubuntu 16.04 and have the following setup:

  • All files for the VM are within a single directory (named after the guest).
  • There is a head.qcow2 symlink that always points to the top "head" qcow2 image. This way I don't have to keep updating the VM config. (virsh edit).
  • The original/bottom image is called base.qcow2

I use the following script to take a snapshot (create a new qcow2 image and make it the head). The script is only executed when the guest is not running.

#!/bin/bash
UNIX_TIMESTAMP=$(date +%s)
CURRENT_BACKING_FILE=`readlink head.qcow2`
NEW_HEAD_FILE="`pwd`/`echo $UNIX_TIMESTAMP`.qcow2"

# Create the new "head" disk image where all future changes are made.
sudo qemu-img create \
  -f qcow2 \
  -b $CURRENT_BACKING_FILE \
  $NEW_HEAD_FILE

# Update the symlink
rm head.qcow2
ln -s $NEW_HEAD_FILE head.qcow2

This will work fine when head.qcow2 points to base.qcow2, or I run the script once so there is only a single backing file. However, If there is a chain of backing files (e.g. a backing file also has a backing file), then I will get the following error:

error: Failed to start domain template-ubuntu-docker2
error: internal error: early end of file from monitor, possible problem: 2018-04-05T19:04:55.597373Z qemu-system-x86_64: -drive file=/media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/head.qcow2,format=qcow2,if=none,id=drive-virtio-disk0: Could not open backing file: Could not open backing file: Could not open '/media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/base.qcow2': Permission denied

The output of qemu-img --backing-chain head.qcow2 is:

image: head.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 196K
cluster_size: 65536
backing file: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/1522954330.qcow2
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/1522954330.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 4.1M
cluster_size: 65536
backing file: base.qcow2 (actual path: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/base.qcow2)
Format specific information:
    compat: 1.1
    lazy refcounts: false
    refcount bits: 16
    corrupt: false

image: /media/ssd_storage2/kvm/vms/templates/template-ubuntu-docker2/base.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 8.8G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

I've played around with deleting images or using qemu-img commit to merge them, and as soon as I go down to just one or two images the guest will start.

Since it looked like a permission error with "could not open" I tried setting permissions to 777 and I still get the same error message, so it shouldn't be a permissions issue.

What do I need to do?

Best Answer

It looks like it's related to AppArmor again (I had a similar issue some years ago on Ubuntu 14.04 that I forgot about). I was able to get it working by:

Installing apparmor-utils

sudo apt-get install apparmor-utils

Then find the libvirt ID in the most recent syslogs:

sudo cat /var/log/syslog | grep "apparmor" | grep "DENIED" | grep libvirt-

Then use that ID with the full path to set it to "complain mode".

sudo aa-complain \
/etc/apparmor.d/libvirt/libvirt-b0cd5bdf-6dfc-4f75-bc0f-e38adff1cdd2

Then start your guest.

This feels like a workaround at best and annoying if I have to do it for each guest individually. I would be happier if someone provided a better solution.

Related Question