Shrinking a Core Storage volume without the booter partition

core-storagedisk-utilityhard drivehfs+partition

I have a 14 TB external drive containing a single 14 TB HFS+ encrypted partition (besides EFI). I cannot resize this partition to add a new one. When I tried using either Disk Utility or diskutil, it says

You can’t perform this resize unless it has a booter (target partition is probably too small)

Essentially I am having the same issue as described in this question: Can't resize partition: "You can't perform this resize unless it has a booter"

However, the answer to that question does not work for my scenario because there is no unallocated space after the single partition and I cannot manually add the booter using gdisk or gpt because of that.

My question is,

  • is there other ways to shrink this partition, besides moving everything out and reformatting it. Or,
  • how do I add in the required booter partition (so that I can resize normally using Disk Utility)?

Note that this is a Core Storage volume because it is encrypted.

  • I had the idea of forcibly creating a new GPT table where there is enough space for the booter, but I haven't done that because I don't know how that would affect the existing partition.
  • I also tried using diskutil cs resizeVolume, which will resize the inner Logical Volume, and then I could add another Logical Volume after it. The problem is that these two volumes will then share the same encryption key AFAIK, which is not desirable.

FYI, here are all the things that have happened that put me in this situation:

  • This is a brand new 14 TB hard drive. I originally formatted it into two partitions: one 10TB and one 4TB. Both were HFS+ journaled encrypted (so they were Core Storage volumes).

  • Later, I attempted to resize them into 12TB/2TB using Disk Utility, which failed miserably.

  • After this, the second 4TB partition could not be found, and the first one could not be mounted. This was reported by diskutil cs info about the first partition (originally 10TB):

    |
+-- Logical Volume Group 69E42F2B-A2E4-41EC-93A9-4D62C0C9082B
|   =========================================================
|   Name:         -none-
|   Status:       Initializing
|   Size:         0 B (0 B)
|   Free Space:   -none-
|   |
|   +-< Physical Volume 4BBFE85D-06C3-4285-98D9-D6D30E3E4031
|       ----------------------------------------------------
|       Index:    0
|       Disk:     disk2s2
|       Status:   Failed
|       Size:     14000175669248 B (14.0 TB)

  • However, I managed to completely recover the first partition (filesystem intact and fsck passed), by destroying and recreating the GPT table, in which the first partition occupies the entire disk — that is, the end of the partition is now right before the secondary GPT. Now the GPT looks like this:

          start         size  index  contents
          0            1         PMBR
          1            1         Pri GPT header
          2           32         Pri GPT table
         34            6
         40       409600      1  GPT part - C12A7328-F81F-11D2-BA4B-00A0C93EC93B
     409640  27344355255      2  GPT part - 53746F72-6167-11AA-AA11-00306543ECAC
27344764895           32         Sec GPT table
27344764927            1         Sec GPT header

  • And now I cannot resize the 14TB partition.

Best Answer

I followed David Anderson's advice and discovered that the partition (as described by GPT) is 3584 bytes (7 LBA blocks) larger than the Core Storage Physical Volume. Using this information, I did manage to shrink the volume to add a second partition.

The process is, however, quite scary: it involves a kernel panic, then some more cryptic, unhelpful, unfixable diskutil errors, and then yet some more frightening failures.

And so let this be an anti-answer:

  • if you are dealing with an Apple-designed filesystem (such as HFS and APFS),
  • and you don't have professional knowledge of the inner workings of them,
  • and you find your volume in an inconsistent state — it should be considered inconsistent as soon as Disk Utility starts complaining —
  • and yet the volume is somehow still usable (you can see your files and such):

Then, for the love of your data and your well-being, instead of trying to fix the problem by meddling with the volume header or partition table (like I did),

Just move everything out and reformat.

I eventually lost no data. Here is my process. I hope that people more knowledgeable than me could point out all the mistakes I made.

Note: 1 block is 512 bytes.

  • (First, I made a copy of the first 512 bytes of the partition, which contains the Volume Header, using dd).

  • Using the gpt command, I learned that my second GPT partition (the main partition) has a size of 27344355255 blocks; following this specification of Core Storage (which is the abstraction layer that allows HFS encryption), I examined the partition volume header and learned that the Core Storage Physical Volume only occupies 27344355248 blocks. This means there was an extra space of 7 blocks.

  • I then decided to free these 7 blocks by recreating the GPT table (using the gpt command), also marking the type of the new partition as Apple Boot.

  • I attempted to create an HFS+ volume on the new partition using diskutil. This is not possible because an HFS+ volume requires a minimum of 512 KB.

  • With the unformatted Apple Boot partition, I made the risky move of asking Disk Utility to resize the 14 TB partition. This time, Disk Utility did not complain about missing booters. However, before the resizing had finished, a kernel panic struck.

  • After restarting, diskutil reported that the original 14 TB partition is now 12 TB, still intact. The 7-block Apple Boot partition now occupied the remaining 2 TB free space, still unformatted.

  • Then, I modified the GPT table again, this time adding a 262144-block Apple Boot partition after the now 12 TB partition, and another Core Storage partition taking up the remaining space.

  • At this point, both the 12 TB and 2 TB partition is showing up in Disk Utility. Upon reformatting the 2 TB partition as a JHFS+ encrypted volume (so that it too has a booter), both partitions can be resized freely using Disk Utility with no errors.

  • The only caveat was that using First Aid (diskutil verifyVolume) on the 12 TB volume now persistently reported the error "Incorrect size for volume" (right after it says the volume "appears to be OK", too). This could not be fixed using any diskutil command. But despite the error, the two volumes can still be resized and normally mounted.

  • At this point, it would seem that my original question has been solved, to a large extent.

This is what the partition scheme now looked like:

   #:                       TYPE NAME                    SIZE       IDENTIFIER 
   0:      GUID_partition_scheme                        *14.0 TB    disk2 
   1:                        EFI EFI                     209.7 MB   disk2s1 
   2:          Apple_CoreStorage - Main Partition -      12.0 TB    disk2s2 
   3:                 Apple_Boot Boot OS X               134.2 MB   disk2s3 
   4:          Apple_CoreStorage Time Machine            2.0 TB     disk2s4 
   5:                 Apple_Boot Boot OS X               134.2 MB   disk2s5

(The rest of this is me not knowing what I am doing.)

  • After this, I came up with the hypothesis that the "Incorrect volume size" error that Disk Utility kept complaining about was about the Logical Volume, and not the Physical Volume.

    • Core Storage has many abstraction layers, the order of which is Physical Volume → Logical Volume Group → Logical Volume Family → Logical Volume.
    • If I am understanding it correctly, PV is the closest to the GPT partition, LVF is where FileVault encryption occurs, and LV is the volume that users can see.

  • To test this out, I deleted the 2 TB partition via Disk Utility. There was no issue. The hard drive now contained EFI, the 14 TB partition, and the 134.2 MB booter.

  • I attempted to use diskutil cs resizeVolume to resize only the Logical Volume back to 12 TB, and not the Physical Volume. The resizing could not proceed because of the "Incorrect volume size" error. (The same went for diskutil cs resizeStack).

  • Then I had the horrible idea to revert not only the GPT table, but also the Core Storage volume header, to the state when this question was posted.

    • This is in the hope that diskutil verifyVolume will then auto-adjust the volume size, and then I can start over the process, but resize the Logical Volume first.
    • I made a backup of the current volume header, then dd rewrote the first 512 bytes of the volume using the backup .img file I made of the header at the start of this answer.
    • After I did that, the 14 TB volume once again was unmountable. All diskutil operations would fail at this point.

  • I attempted to revert this change by writing the 512-byte volume header back. Then, another kernel panic occurred.

  • This time, things got much more serious. For the next few hours, every time I connected the hard drive to my laptop, macOS immediately kernel panicked — whether it was in normal mode, Recovery, Safe Mode, or Single User Mode.

  • I tried booting into Ubuntu from USB, and examine and manipulate the Volume Header from there, to no avail.

  • I almost lost hope, when I remembered that, back when the partition first failed, I ran dd through the first few GBs of the partition and saved it as an image, which was since deleted. I recovered it using Disk Drill, and after painfully comparing hex dumps, I decided to copy the first 62500 blocks (32 MB) of the image file back to the partition.

    • My idea was that this would cause macOS to recognize the volume in the same way that it recognized the original 14 TB volume when it first failed, and then if I redo the steps I described in the question (recreating the GPT table), I could get the volume back a second time, and I thought that the worst that could happen is I would lose all file changes since the partition first failed.

  • Shockingly, not only did this recover the partition, but all the files that were on the hard drive before I broke the partition the second time are still there.

  • Now I am in the process of backing up everything so that I can start the hard drive fresh.

To reiterate: when in the slightest doubt, backup!

Related Question