MacOS – Partition table error in external ARECA RAID

The TestDisk results for macOS formatted disks are/were often misleading or even wrong because the algorithm misinterprets special hidden volume content. AFAIK it tries to detect special empty blocks (2) followed by the occurrences of the strings H+ and/or HFSJ in the third block - which marks the beginning of a HFSJ volume. A similar sequence is used to detect the last blocks of a volume. Such 1536 Byte "blocks" are more frequent than TestDisk can handle.

I prefer a manual approach.

Preconditions:

• GPT (GUID Partition Table)
• some "standard" partitioning with Disk Utility previously (the disk may contain several volumes though)
• the disk mustn't be initialized after the incident
• the disk or volume mustn't be repaired (with Disk Utility, Disk Warrior or similar tools)

Methodology:

• determine the device block size
• determine the volume type of the vanished/missing volume
• read-out the volume header (and sometimes the second volume header) to determine block size and number

Basics:

First get all available disks and the partition table of the broken disk (here disk5):

• diskutil list
• diskutil umountDisk disk5
• sudo gpt -r show disk5

Device Block Size:

To determine the block size of the disk enter:

diskutil info disk5 | grep "Device Block Size"

Depending on the Device Block Size (either 512 B or 4096 B) and the total size of the disk they either contain a 200 MiB or 300 MiB EFI partition as first partition after the partition table, then the main volume and finally some free space or a second partition and some free space. The last few blocks are occupied by the second ("backup") partition table.

The standard Apple GUID partition scheme of a 512 B disk looks like this:

                0           1         PMBR
                1           1         Pri GPT header
                2          32         Pri GPT table
               34           6         
               40      409600      1  GPT part - C12A7328-F81F-11D2-BA4B-00A0C93EC93B
           409640  part2-size      2  GPT part - partition type
         - depending on the formatting of the disk a 3rd partition and/or free space -
    total_size-33          32         Sec GPT table
     total_size-1           1         Sec GPT header

The standard Apple GUID partition scheme of a 4096 B disk looks like this:

               0           1         PMBR
               1           1         Pri GPT header
               2           4         Pri GPT table
               6       76800      1  GPT part - C12A7328-F81F-11D2-BA4B-00A0C93EC93B
           76806  part2-size      2  GPT part - partition type
         - depending on the formatting of the disk a 3rd partition and/or free space -
    total_size-5           4         Sec GPT table
    total_size-1           1         Sec GPT header

512 B disks may also have a 614400 blocks(512) EFI partition (partition 1) or 4096 B disks may have a 51200 blocks(4096) EFI partition - both can be found much less often than the other way round though. The size of the main partition is reduced or enlarged respectively.

An external RAID with a RAID-controller may contain a bunch of 4096 B disks but will still present the RAID-volume as 512 B device!

Volume Type:

To determine the volume type you have to hexdump the first blocks after the EFI partition.

This is done by piping data stream input in dd to hexdump:

sudo dd if=/dev/diskX | hexdump

To hexdump a 512 B disk (with a 200 MiB EFI) use the following command:

sudo dd if=/dev/disk5 skip=409640 bs=512 count=3 | hexdump

To hexdump a 4096 B disk (with a 300 MiB EFI) use the following command:

sudo dd if=/dev/disk5 skip=76806 bs=4096 count=1 | hexdump

In both cases a normal HFS+ volume is indicated by its signature 0x482b (="H+") at 0x0000400 and/or by its lastMountedVersion 0x4846534a (="HFSJ") starting at 0x0000408

0000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
*
0000400→48 2b←00 04 80 00 20 00→48 46 53 4a←00 00 2b ab
...

In both cases an APFS volume is indicated by 0x4e585342 (= APFS magic string) starting at 0x0000020:

0000000 0f d5 99 82 f1 8b e4 67 01 00 00 00 00 00 00 00
0000010 14 33 00 00 00 00 00 00 01 00 00 80 00 00 00 00
0000020→4e 58 53 42←00 10 00 00 f6 37 b9 03 00 00 00 00
...

Volume Header:

Both dumps also contain the Allocation Block Size and the Number of Allocation Blocks of the (first) volume or the APFS container. Both values determine the size of the (lost) volume/container. The allocation block size mustn't be confused with the device block size!

The allocation block size of an HFS+ volume can be found in the volume header. In the dump above the volume header starts at 0x0000400 and ends at 00005FF.

The allocation block size is an UInt32 starting at 0x0000428

0000400 48 2b 00 04 80 00 20 00 48 46 53 4a 00 00 2b ab
0000410 d2 c3 e0 e6 d8 cd 44 73 00 00 00 00 d2 c3 d2 d6
0000420 00 01 c9 aa 00 00 08 8d→00 00 20 00←2b a9 28 fa
0000430 02 56 52 7a 2b 74 ff f4 00 01 00 00 00 01 00 00

The allocation block number is an UInt32 starting at 0x000042C

0000400 48 2b 00 04 80 00 20 00 48 46 53 4a 00 00 2b ab
0000410 d2 c3 e0 e6 d8 cd 44 73 00 00 00 00 d2 c3 d2 d6
0000420 00 01 c9 aa 00 00 08 8d 00 00 20 00→2b a9 28 fa←
0000430 02 56 52 7a 2b 74 ff f4 00 01 00 00 00 01 00 00

To convert hex to decimal use echo "obase=10; ibase=16; (uppercase hex!)" | bc

So the allocation block size of the above example is

echo "obase=10; ibase=16; 00002000" | bc (=8192 byte)

and the number of allocation blocks is

echo "obase=10; ibase=16; 2BA928FA" | bc (=732506362)

The total size of the example volume is 732506362 x 8192 Byte (= 6000692117504 Byte or 11720101792 blocks(512) or 1465012724 blocks(4096)).

The block size and the block number of an APFS container can be found in the container superblock which starts at 0x0000020.

The block size is an UInt32 (reversed) starting at 0x0000024

0000000 0f d5 99 82 f1 8b e4 67 01 00 00 00 00 00 00 00
0000010 14 33 00 00 00 00 00 00 01 00 00 80 00 00 00 00
0000020 4e 58 53 42→00 10 00 00←f6 37 b9 03 00 00 00 00
0000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...

The block number is an UInt64 (reversed) starting at 0x0000028

0000000 0f d5 99 82 f1 8b e4 67 01 00 00 00 00 00 00 00
0000010 14 33 00 00 00 00 00 00 01 00 00 80 00 00 00 00
0000020 4e 58 53 42 00 10 00 00→f6 37 b9 03 00 00 00 00←
0000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...

To convert hex to decimal with echo... command you have to reverse the byte order:

(00 10)⬄(00 00) > 00⬄00 00⬄10 > 00 00 10 00
and
(f6 37 b9 03)⬄(00 00 00 00) > (00 00)⬄(00 00) (f6 37)⬄(b9 03) > 00⬄00 00⬄00 b9⬄03 f6⬄37 > 00 00 00 00 03 b9 37 f6

So the block size of the above example is

echo "obase=10; ibase=16; 00001000" | bc (=4096 byte)

and the number of blocks is

echo "obase=10; ibase=16; 0000000003B937F6" | bc (=62470134)

The total size of the example volume is 62470134 x 4096 Byte (= 255877668864 Byte or 499761072 blocks(512) or 62470134 blocks(4096)).

Recreating the lost volume:

Depending on your findings in the previous steps (512|4096)/(HFS+/AFPS)/(block size & block number) add the missing EFI and the main volume to the partition table.

Examples with the above example disks (and both missing EFI and main partition):

1. 512 B disk, 200 MiB EFI & APFS container (499761072 blocks(512))

   sudo gpt add -i 1 -b 40 -s 409600 -t C12A7328-F81F-11D2-BA4B-00A0C93EC93B /dev/disk5
   sudo gpt add -i 2 -b 409640 -s 499761072 -t 7C3457EF-0000-11AA-AA11-00306543ECAC /dev/disk5
   

2. 4096 B disk, 300 MiB EFI & HFS+ volume (1465012724 blocks(4096))

   sudo gpt add -i 1 -b 6 -s 76800 -t C12A7328-F81F-11D2-BA4B-00A0C93EC93B /dev/disk5
   sudo gpt add -i 2 -b 76806 -s 1465012724 -t 48465300-0000-11AA-AA11-00306543ECAC /dev/disk5
   

Verify the final volumes with diskutil verifyVolume diskXsY.

---

If you're faced with a 512 B disk and a 300 MiB EFI (or a 4096 B disk and a 200 MiB EFI) which will result in hexdump gibberish, you have to slightly apply the dd ... | hexdump command.