Linux – How to Correct 512-byte Sector MBR on a 4096-byte Sector Disk

hard drivelinuxmbrpartitioning

Final update:

I already knew what I needed to do to fix this problem; I just didn't know how to do it. I was hoping there would be some ready-made tool to do that automatically – but couldn't find any. I am accepting Rod's answer because despite not directly solving my issue, it gives a very good background on the sector size issue, and gave me confidence that the issue really was partition alignment and addressing. For those that come to this question having the same issue, read it thoroughly and carefully, including comments, before doing anything.

In the beginning

I had a computer and needing more space I've bought a new 500GB drive and an USB enclosure. Soon I have noticed that if I partitioned the drive on the enclosure and moved it to the computer, it wouldn't recognize the partitions (and vice-versa). I assumed it was a problem with the enclosure and didn't worry about it.

Then, tragedy

A wonderful day, my computer decided to not turn on anymore. Turns out the motherboard (unbranded, just a big MADE IN CHINA printed on it) is dead. I've been using it as a file-server and that 500GB drive is now full of data that I can not afford to lose. I am broke now and can not afford a new computer, so my only hope was the "defective" USB enclosure.

The investigation

Armed with several Linux distributions, a laptop, VirtualBox and the enclosure I did a forensic analysis on the issue. dmesg reported partition size was beyond end-of-drive. So I went through hard drive datasheets, calculated sector counts from scratch, tested drive boundaries manually with dd, and everything looked OK, until I fired up fdisk and it said:

    Note: Sector size is 4096 (not 512).

How modest of fdisk. This "note" was the root of all the issues. After some more fiddling these conclusions were drawn:

  • The USB enclosure is not defective.

  • The SATA controller on the now dead motherboard is the one that was "weird", at least. It did not report 4096-byte sectors to the operating system, so the OS happily created the MBR using 512-byte sector addresses.

  • Now when I try to access the partition, the OS tries to use the 512-byte based addresses on a 4096-byte sector drive, and of course, it's not gonna work.

The question

  • So, how can I correct the addresses in the MBR so they are valid on a 4096-byte sector size, aside from manually editing the MBR on an hex-editor, and

  • The partitions are not aligned for 4096-byte sectors. There is some tool available to align them aside from copying in and out of another drive? (I do not have spare drives), or will I need to create some tool that "shifts" the data to the side a little chunk at a time? Partitions are ext3.

Thanks!

Update:

I found there is a clever way to use dd to shift the partition in-place in this question: How to move a partition in GNU/Linux?
But I don't know if it will work on a slice of a sector, though. I can't test it right now but will do when I have some time.

Update 2:

So I have successfully aligned the partition using the method above and hand-edited the MBR on a hex editor. As soon as I re-plugged the HDD, boom partition automatically mounted! I do not recommend this though, there were I/O errors during the process and I could have lost everything, see comment on Rod's answer. For the other partition I will not take risks and will use an old HDD and align chunks at a time by copying the data and then pasting it back on a different position.

Best Answer

Sector-size issues are becoming quite complex. Until late 2009, the vast majority of hard disks used 512-byte sectors, and that was that. In late 2009, disk manufacturers began introducing so-called Advanced Format (AF) disks, which use 4096-byte sectors. These first AF disks (and, AFAIK, all AF disks today) present an interface to the computer that shows each 4096-byte physical sector as being split up into eight 512-byte logical sectors. This conversion enables older tools, including many BIOSes, that were built with 512-byte assumptions, to continue to work. I don't know if your disk uses AF or not, but in either case, it almost certainly uses a 512-byte logical sector size, meaning that the interface to the OS should use 512-byte sectors.

Complicating matters is certain USB disk enclosures. Some of these enclosures do the reverse of what AF does: They take eight disk sectors and bundle them into one new 4096-byte sector. I'm not sure what the reasoning is behind this move, but one practical advantage is that disks larger than 2TiB can be used with the old MBR partitioning system. One major disadvantage is that a disk partitioned in one of these enclosures can not be used directly or in an enclosure that doesn't do this type of translation. Likewise, a disk prepared without this translation can't be used when it's transferred into such an enclosure. Note that this problem goes well beyond the MBR itself; your disk might identify the first partition as beginning on (512-byte) sector 2048, but if your OS were to seek to (4096-byte) sector 2048, it would not find the start of that partition! You've run into this problem. As such, your initial thought that it's the USB enclosure's fault is closer to the mark than your more recent thought that your motherboard messed it up. I've never heard of a motherboard translating sector size in this way. (Some hardware RAID devices do so, though.)

I don't know of a way to force Linux to adjust its idea of the sector size, but if you have enough disk space, doing a low-level disk copy to another disk may help. For instance:

dd if=/dev/sdb of=~/image.img

This will copy your disk from /dev/sdb (the USB disk; adjust as necessary) to the file ~/image.img. You can then use the following script to mount the image's partitions:

#!/bin/bash
gdisk -l $1 > /tmp/mount_image.tmp
let StartSector=`egrep "^   $2|^  $2" /tmp/mount_image.tmp | fmt -u -s | sed -e 's/^[ \t]*//' | head -1 | cut -d " " -f 2`

let StartByte=($StartSector*512)

echo "Mounting partition $2, which begins at sector $StartSector"

mount -o loop,offset=$StartByte $1 $3

rm /tmp/mount_image.tmp

Save the script as, say, mount_image and use it like this:

./mount_image ~/image.img 2 /mnt

This will mount partition 2 of image.img to /mnt. Note that the script relies on GPT fdisk (gdisk), which most distributions include in a package called gptfdisk or gdisk.

In the long run, a better solution is to find a way to connect the disk that won't do the sector-size translation. A direct connection to a new motherboard should do the trick; or you can probably find an external enclosure that doesn't do the translation. In fact, some enclosures do the translation on USB ports but not on eSATA ports, so if your enclosure has an eSATA port, you could try using that. I realize that these solutions are all likely to cost money, which you say you don't have, but maybe you can trade your translating enclosure for one that doesn't do the translation.

Another option that occurs to me is to try using a virtual machine like VirtualBox. Such a tool might assume a 512-byte sector size when accessing the disk device, effectively undoing the translation; or you might be able to copy the disk's contents raw (as in dd if=/dev/sdc of=/dev/sdb) within the virtual machine, which might copy the contents with compression, thus enabling the image to fit on less disk space than the original consumes.

Related Question