I'm testing an MDM solution (with an in house Server.app MDM instance), which enforces FileVault. It's setup with both an Institutional Recovery Key (IRK) and a Personal Recovery Key (PRK). The last one is also saved to MDM.
Basically, we generated a keychain like this, exported the certificate from the keychain, and added it to an MDM profile (we're using Server.app's MDM).
It's working great. When I do a diskutil apfs listCryptoUsers diskNxM, I get all the users I expect, including an user of type Institutional Recovery User and one with type Institutional Recovery External Key. Resetting password via Open Directory works fine. Unlocking the drive with the PRK works fine.
And now I'm testing to unlock the volume with the said IRK.
I booted via Recovery (cmd+R), and when I execute diskutil apfs unlockVolume /dev/diskNsM -recoveryKeyChain /Volumes/RecoveryDrive/FileVaultMaster.keychain (the keychain is unlocked, and the drive is correct), I get this error:
Error unlocking APFS Volume: The external-to-APFS security system's credential-unwrap
operation failed (-69534)
I checked unlocking the keychain using a different password, and that failed directly. Someone here suggested to remove the certificate from the keychain. This didn't work either. I double checked the volume was APFS. It was.
Any idea's (besides creating a new FileVault Master keychain and do the whole process one more time)?
Best Answer
The thing I forgot to mention was I didn't generate the Keychain myself. I started off with a keychain with a certificate and private key that I could unlock, and an MDM profile with the exported certificate.
Because I couldn't find another solution, I created a new Keychain. I used this Apple Support Document. I didn't remove the private key, because I deploy it by exporting the certificate, and distributing it via MDM.
However, I did use the "Use the private key to unlock a user's startup disk" procedure from this document to unlock the volume and it worked! ?
So this is not an actual answer to the asked question, but certainly a solution. I double checked the computer encrypted with the old keychain certificate with this procedure, but got the same error message. So probably there was something wrong with the keychain and/or certificate...