Do hard disk drives turn on 512e (512byte emulation of 4k sectors) as-needed depending on host controller

external hard drivehard drivesatausb

I have a 4 TB hard disk that has 4k physical sector size (Advanced Format) and I attached it to two SATA host controllers (one internal and one in a USB enclosure). On the (older, about 2010) it comes up as 512B logical sector size, but reports its physical sectors are 4096 bytes. In the recent USB enclosure, it is reported as having 4096B logical and physical size:

# internal host controller
sd 4:0:0:0: [sdd] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB)
sd 4:0:0:0: [sdd] 4096-byte physical blocks

# USB enclosure
sd 18:0:0:0: [sdd] 976754646 4096-byte logical blocks: (4.00 TB/3.63 TiB)

The "internal" controller shows behavior I had previously seen, which is usually called "512e" where the drive's firmware emulates 512 sector LBA addressing even though it does not physically write sectors that way. Operating systems (and admins) can ensure that partitions are aligned such that (small) write commands can be grouped so that the disk can overwrite full sectors instead of falling back to writing to part of a physical sector, where it needs to read the remainder of content first.

However, seeing "4k native" behavior on a drive (USB enclosure) was new to me and my initial thought was the enclosure is emulating 4096B addressing on top of the drive's emulated 512B emulation.

The only occurrences of this that I could find with a Web Search where USB storage facilities that registered with 4k logical sectors. I presume they emulate 4k over 512e so allow for MBR partition tables so that large drives can be used on legacy/embeeded devices (Smart TVs and what not) that only support MBR+FAT32 usb mass storage.

After contacting the manufacturer, they claimed the drive is actually operating in non-emulated (native) 4k mode, the USB SATA controller is also doing no emulation at all. This would require the drive's firmware to detect whether the host controller supports (desires?) addressing sector sizes of 4KB.
I could not find anything about this in the public documentation of SATA standards. So I ask:

Has anyone seen "4096-byte logical blocks" on their SATA controller?
Do drives actually support enabling/disabling emulation on demand? If so,
- How does this determination work?
- Can you override it on the drive via a flag?
- Can you override it on the host controller via a driver/flag?

Best Answer

I've seen a few 4Kn SATA drives working in the hard drive validation industry, but I wasn't aware they were shipping any to customers, due to the limited hardware and software support for them and lack of customer demand.

There are two competing mappings for physical vs. logical sector sizes. The Long Logical Sector feature allows a device to have logical sectors longer than 512B (e.g. 4Kn), and the Long Physical Sector feature allows a device to have multiple logical sectors per physical sector (e.g. 512e), though they are not necessarily mutually exclusive. If the manufacturer is claiming the drive is 4Kn, the older controller may providing an emulation layer, like read-modify-write, for older applications and hardware, as older systems simply don't support the 4Kn format.

The sg_sat_identify command from the sg3_utils package would probably give you the information you need (check words 106-108 for Physical sector size/logical sector size information).

It doesn't look like SATA drives can support emulation on-demand, per se, but if the drive is indeed using 512e, instead of 4Kn, performance optimization is maximized by the driver (or possibly USB SATA controller) limiting reads/writes to 4K boundaries, e.g. having the transfer start on an LBA where the lower 3 bits are 0, and end on an LBA where the lower 3 bits are 1:

start_lba & 0x3 == 0
end_lba & 0x3 == 1

So, basically, there's no "switch" that can tell the drive to start or stop emulation. From an end-user perspective, your best bet is probably to make sure your drives are using the best file system for whichever mapping your drive uses and to make sure it's aligned correctly. The following site gives a pretty good breakdown of the common file systems and their stats on performance vs. block alignment on different file systems: http://www.ibm.com/developerworks/library/l-linux-on-4kb-sector-disks/index.html

The 512-byte emulation is intended for compatibility with older systems. However, writes involving only part of a physical 4K sector can cause reduced performance because the sector needs to be read and modified before it can actually be written.

When a legacy operating system tries to write to an Advanced Format disk, performance issues can arise because the logical sectors written may not match up with the physical sectors.

When only part of a 4K physical sector is read, the data is simply read off the physical sector and there is no reduction in performance. However, when the system tries to write to part of a physical sector (e.g. an emulated 512-byte sector rather than the whole physical sector), the hard drive needs to read the whole physical sector, modify the changed portion in the hard drive's internal memory, and write it back to the platters. This is called read-modify-write (RMW), an operation which requires an extra rotation of the disk and therefore reduces performance. Seagate explains this as follows:

[...] the hard drive must first read the entire 4K sector containing the targeted location of the host write request, merge the existing data with the new data and then rewrite the entire 4K sector:

In this instance, the hard drive must perform extra mechanical steps in the form of reading a 4K sector, modifying the contents and then writing the data. This process is called a read-modify-write cycle, which is undesirable because it has a negative impact on hard drive performance.

Disk partitions that are not aligned to a 4K boundary can cause degraded performance as well.

Traditionally, the first partition on a hard disk starts at sector 63. Windows XP and older operating systems partitioned disks in this manner. Newer versions of Windows will create partitions on a 1 MB boundary, ensuring proper alignment to the physical sectors. This is called Alignment 0.
- This odd number is an artifact of cylinder-head-sector (CHS) addressing used in INT 13h, the legacy BIOS API used for disk access. On legacy systems and bootloaders that used the INT 13h API, all partitions must begin and end on cylinder boundaries. Even after logical block addressing (LBA) was introduced, fake CHS values (which did not correspond to the actual disk geometry) were used to maintain compatibility with the legacy API. Since CHS addressing originally supported a maximum of 63 sectors per cylinder, the first partition would begin on sector 63. Windows XP (before Service Pack 3) and earlier versions of Windows will not boot if the system volume isn't on a cylinder boundary.
Because LBA 63 is not a multiple of 8 (eight 512-byte legacy sectors fit into a 4K sector), an Advanced Format disk which is formatted in the old manner will have clusters (the smallest unit of filesystem data allocation, typically 4K in size) that are not aligned to the physical sectors on a 4K disk, a condition called Alignment 1. As a result, an I/O operation that otherwise involves 4K of data now spans two sectors leading to a read-modify-write operation that reduces performance.

While information about physical sector size is unnecessary if the OS always writes data on a 4K boundary, this information may still be needed by applications which perform low-level I/O.

When a drive reports that its physical sector size is 4K, the OS or application can tell that it is an Advanced Format drive and therefore must avoid performing I/O operations that do not span full physical sectors. A drive that reports 512-byte native sectors does not impose this restriction. While newer operating systems will usually try to read or write data in 4K units whenever possible (making this information irrelevant), applications which perform low-level I/O may need to know the physical sector size so that they can adjust accordingly and avoid misaligned or partial-sector writes that cause slow RMW cycles.

Your SSD provides the ability to change the reported physical sector size because it is necessary for compatibility with certain storage arrays.

Datacenters often have storage arrays consisting of legacy 512n drives. 4K drives, even those that emulate 512-byte sectors, may not be compatible with such arrays, so this feature is necessary to ensure compatibility. See this forum thread:

We can't just stick a 4K drive in an array formatted with 512b disks. Many arrays (most notably ZFS based storage, which is increasingly popular as software defined storage makes waves) will not accept a replacement drive with a different physical sector format.

Note that better performance will be attained on modern systems if the drive is configured to use 4K sectors.

Best Answer

Related Solutions

Linux – Logical Sector Size Changes with USB vs SATA

Hard Drive – Why Report Physical Sector Size?

The 512-byte emulation is intended for compatibility with older systems. However, writes involving only part of a physical 4K sector can cause reduced performance because the sector needs to be read and modified before it can actually be written.

Disk partitions that are not aligned to a 4K boundary can cause degraded performance as well.

While information about physical sector size is unnecessary if the OS always writes data on a 4K boundary, this information may still be needed by applications which perform low-level I/O.

Your SSD provides the ability to change the reported physical sector size because it is necessary for compatibility with certain storage arrays.

Related Question