Disk image of OS + Recovery partition

disk-utilitydiskutildmg

I'm working with multiple OS X versions frequently and I'd like to have an image which contains a particular OS version's system partition and its recovery partition. Is this possible? (the disk where these partitions are, contains several other partitions which I don't want, so full-disk image is not an option)

I'm guessing this cannot be done as a single-step process, so I've already created an appropriately-sized sparsebundle image with GPT layout:

hdiutil create -size 35g -layout GPTSPUD -type SPARSEBUNDLE /path/to/new_image.dmg

The next step would be to copy the relevant partition onto this image, but I don't know how to do this. My Linux experience tells me the partitioning utility should be the place to look for this, but I don't see anything in diskutil's manual which looks like it does what I want.

Hints/ideas?

Best Answer

Starting from a working setup

A working setup is disk, or full-disk image with multiple partitions (FDI hereafter), where the OS and its recovery partition is known to be correct, most likely because it was created by the OS Installer. As mentioned in the comment, asr is enough to handle copying both partitions in this case.

Assuming the source OS and recovery partitions are disk3s11 and disk3s12, respectively, I recommend the following procedure:

Shrink the OS partition to its minimum size. This isn't necessary, but I suggest doing it when you can never be sure if the user's target partition will be as big as the one you're using.
```
$ diskutil resizeVolume disk3s11 limits

... (minimum size will be mentioned here) ...

$ diskutil resizeVolume disk3s11 24GB
```
Create appropriately-sized temporary FDI as a sparsebundle. Remember, the Recovery takes extra 650 MB over the system partition, and EFI takes another 210 MB. So I'll add an extra GB to the size used in step 1 just to be safe. The catch here is, hdiudil support only binary prefixes (1024 for k, etc.). In this example, 25 GB = 23.29 GiB.
```
$ hdiutil create -layout GPTSPUD -type SPARSEBUNDLE -size 23.29g /tmp/temp_FDI

created: /tmp/temp_FDI.sparsebundle
```

Attach the image without mounting it (it would fail, since the partition was not formatted). Take note of the /dev/ entry for the Apple_HFS partition.

$ hdiutil attach /tmp/temp_FDI.sparsebundle -nomount

/dev/disk5              GUID_partition_scheme           
/dev/disk5s1            EFI                             
/dev/disk5s2            Apple_HFS

Run asr with the image's empty partition as the target. You'll have to enter full path to /dev/... entries. Also, since the source is a physical disk, elevated privileges are necessary. You should see the tool finish with 2 rows of Restoring... progress.

$ sudo asr restore --source /dev/disk3s11 --target /dev/disk5s2 --erase  --noverify

    Validating target...done
    Validating source...done
    Erase contents of /dev/disk5s2 ()? [ny]: y
    Validating sizes...done
    Restoring  ....10....20....30....40....50....60....70....80....90....100
    Restoring  ....10....20....30....40....50....60....70....80....90....100

Finally, create a compressed read-only image from the sparsebundle. Use the sparsebundle's "whole disk" /dev/ entry (/dev/disk5).

$ hdiutil create -format ULFO -srcdevice /dev/disk5 -o /tmp/OS_X_10.11-combined_FDI2.dmg

...
Elapsed Time:  2m 33.743s
File size: 8000565446 bytes, Checksum: CRC32 $5D496986
Sectors processed: 48842670, 38032196 compressed
Speed: 120.8Mbytes/sec
Savings: 68.0%
created: /tmp/OS_X_10.11-combined_FDI2.dmg

You may wish to use UDZO format if you need your image to be compatible with older versions of OS X and their respective Recovery environments (ULFO was only introduced in 10.11).

Working from the ground up

It is possible to assemble a "working setup" from single-partition images (images which don't have partition table, and therefore no hint for asr that a particular partition is a Recovery; SPI hereafter). The procedure is not for the faint of heart.

I'm going to assume that I have a compressed, read-only SPIs of both the system partition, and the recovery partition. Shrinking the partitions to minimum size is a whole another can of worms, where using a full-blown UDRW image seems unavoidable. Feel free to suggest a simpler solution.

With that in mind, do a pre-shrink of the system partition image. Readonly image can only be shrunk with a shadow.
```
$ hdiutil resize -sectors min  OS_X_10.11-system.dmg -shadow /tmp/pre-shrink.shadow
```

Check the image size, then attach it

$ hdiutil imageinfo OS_X_10.11-system.dmg -shadow /tmp/pre-shrink.shadow

...
    Total Bytes: 27648868352

$ hdiutil attach OS_X_10.11-system.dmg -shadow /tmp/pre-shrink.shadow -nomount

...
/dev/disk7

The size is about 27.65 GB.

Create a sparse image big enough to hold the system and recovery partitions in full size - AFAICT, SPIs cannot be resized and that must be done inside an FDI. With extra space needed for EFI and recovery, we'll need 28.6 GB = 26.64 GiB.

$ hdiutil create -layout GPTSPUD -type SPARSEBUNDLE -size 26.64g /tmp/temp_FDI
$ hdiutil attach /tmp/temp_FDI.sparsebundle -nomount
/dev/disk5              GUID_partition_scheme           
/dev/disk5s1            EFI                             
/dev/disk5s2            Apple_HFS

Do an asr restore from the system SPI's /dev/ node into the sparsebundle's data partition.
```
$ asr restore --source /dev/disk7 --target /dev/disk5s2 --erase --noverify
```

Now, check the minimum size of the partition using diskutil. Then use resizeVolume to shrink the partition and create the recovery partition in a single step.

$ diskutil mount disk5s2
$ diskutil resizeVolume disk5s2 limits

... (minimum size will be mentioned here) ...

$ diskutil resizeVolume disk5s2 24GB JHFS+ Recovery %recovery% Free\ Space dummy 1m

Restore the recovery SPI into the new partition on the FDI. It will probably get mounted, so unmount it. You can work with the image directly.
```
$ asr restoreexact --source /path/to/OS_X_10.11-recovery.dmg --target /dev/disk5s3 --erase --noverify

...
$ diskutil umount disk5s3
```
Here's the tricky bit - dump the partition table using the gpt utility, taking note of the recovery partition's start and size values. Delete the record for the recovery partition, and re-add it with modified type GUID. You will know which record is for the recovery partition by the index ("disk5s3"), and by its size (shown here in 512 B sectors).
```
$ gpt show disk5
...
   47546784    1269536      3  GPT part - "Recovery"
...
$ gpt remove -i 3 disk5
$ gpt add -b 47546784 -s 1269536 -t 426F6F74-0000-11AA-AA11-00306543ECAC disk5
```
Optional: I also recommend changing the partition name (set to "disk image" by default) to match the system partition's volume name.
```
$ gpt label -i 2 -l "OS X 10.11" disk5
```
Now, since hdiutil resize doesn't work with sparsebundles the way we'd need, you can either:
- Leave it at that. It's not like the free space consumes anything. Continue with step 5 in the Working setup scenario.
- Convert the temp_FDI.sparsebundle image to UDRW, and try to properly shrink that. I haven't tested that and the need to write out the entire size image for this step makes the whole exercise pointless.
- Treat the sparsebundle existing at this point as a disk with optimally-shrunk system, and perform the first scenario on it, skipping step 1. Ie. create another sparsebundle image, and run asr, which will this time copy both partitions into the new image, and convert that to ULFO image.

Restoring

Steps for restoring from the created image:

Mount the image using

# hdiutil attach /tmp/backup_image.dmg -nomount

Start the restore, either by:
- Use the "Disk Utility" app, restoring from the partition in the backup_image to the partition on your drive (the one in Sierra just works, restoring the Recovery partition too: you will see two Restoring/Verifying steps)
- Use asr, similarly as described above:
  
  $ asr restore --source /dev/disk1s2 --target /dev/disk0s4 --erase --noverify

Related Solutions

How to delete a partition (coreStorage logical volume?) from the Terminal

I highly recommend completely backing up the machine before attempting this, either using TM with no exclusions set, or better yet, cloning the whole drive using a program like Carbon Copy Cloner.

Unmount the Logical Volume:

sudo diskutil unmount force /dev/disk1

Remove the Logical Volume Group and all of its contents:

sudo diskutil cs deleteLVG B6308EC8-297D-44BD-9212-6BD867F6331B

From diskutil's man page:

  delete | deleteLVG lvgUUID | lvgName
           Delete a CoreStorage logical volume group. All logical volume families with their
           logical volumes are removed, the logical volume group is destroyed, and the now-
           orphaned physical volumes are erased and partition-typed as Journaled HFS+.

Merge the now orphaned partition disk0s4 into startup volume disk0s2 (note: disk0s3, one recovery partition, will be erased and merged as well). disk0s5, the other recovery partition, shouldn't be affected:

sudo diskutil mergePartitions HFS+ "Apple_HFS Macintosh HD" disk0s2 disk0s4

Again, from the man pages:

 Merge two or more partitions on a disk.  All data on merged partitions other than the first
                will be lost.  Data on the first partition will be lost as well if the force argument is
                given.

                If force is not given, and the first partition has a resizable file system (e.g. JHFS+), the
                file system will be preserved and grown in a data-preserving manner; your format and name
                parameters are ignored in this case.

Verify the boot Volume structure.

diskUtil VerifyVolume /

Verify the partition map:

diskutil verifyDisk disk0

Boot into Recovery Mode, or Internet Recovery if needed -- if any errors appear:

You should be left with:

/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *320.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS Macintosh HD            319.7 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3

Disk Utility reading disk incorrectly; viewing ext4 as JHFS+ and seeing Recovery HD

If you want better answers, try refining your question. Also, post a comment after my answer, so I will be notified.

Question:

1) Recovery HD is visible, which, I could be mistaken, but this shouldn't be visible OR mounted in Disk Utility

Answer:

In the MBR partition table, the id should be AB and you have AF. In the GPT, the partition type should be 426F6F74-0000-11AA-AA11-00306543ECAC and you have48465300-0000-11AA-AA11-00306543ECAC. This is why it is visible.

Question:

2) disk0s4 is actually ext4, NOT JHFS+.

Answer:

You used the gdisk to change the partition type code for disk0s4 from AF00 to 8300 while disk0s4 while it was still mounted. The Disk Utility application will not see the change until you unmount and mount disk0s4.

Question:

where is disk0s5

Answer:

Currently, the fifth entry in the GPT is not being used. You have not given any information regarding the values in this table entry, other than due to its absence the partition type must be 00000000-0000-0000-0000-000000000000. For all anyone knows, the UUID, attributes, label and starting/ending sector numbers may still be stored in the fifth table entry.

Note: I have observed the Disk Utility application requires the entries in the GPT to be in ascending order and contiguous. The output of the gpt command shows the indices to be in ascending order, but since the fifth entry is missing, the indices are not contiguous. When a situation like this occurs, the Disk Utility application can incorrectly display the contents of the GPT. In your case, the errors shown by the Disk Utility application where most likely caused by the use of gdisk.

Why did you use a BIOS/MBR install of Ubuntu? I know you can download such an iso, but the EFI/GPT iso version also works on Mac's. This is what I choose for my mid 2007 20-inch iMac. This way you can avoid using a hybrid GPT.

Best Answer

Starting from a working setup

Working from the ground up

Restoring

Related Solutions

How to delete a partition (coreStorage logical volume?) from the Terminal

Disk Utility reading disk incorrectly; viewing ext4 as JHFS+ and seeing Recovery HD

Related Question