Linux – Why would a 3TB disk show as being 746GiB

hard drivelinuxUbuntu

I have a 3TB HDD and Ubuntu 14.04 live DVD is showing it as being 746.5GiB. I've seen this sort of problem with other disks in other situations before and never found a solution. I've even had Seagate replace a HDD telling me they were unable to fix the HDD after a disk imaging tool somehow permanently told the HDD it was a smaller disk.

Two questions:

  • How did this happen?
  • How do I fix it?

HDD Background for this case

  • The disk used to be part of a ZFS RAIDZ using the bare disk instead of a partition.
  • it sat on a shelf for a quite a few months.
  • I used GParted to try and delete everything on the disk

I'm currently running the command 

dd if=/dev/zero of=/dev/sdd bs=16M

Before that I ran:

root@ubuntu:/home/ubuntu# dd if=/dev/zero of=/dev/sdd bs=10M count=128
128+0 records in
128+0 records out
1342177280 bytes (1.3 GB) copied, 42.8214 s, 31.3 MB/s

root@ubuntu:/home/ubuntu# smartctl -i /dev/sdd
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.13.0-32-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Western Digital Caviar Green (AF, SATA 6Gb/s)
Device Model:     WDC WD30EZRX-00DC0B0
Serial Number:    {blanked}
LU WWN Device Id: 5 0014ee 0036bc22d
Firmware Version: 80.00A80
User Capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 1.5 Gb/s)
Local Time is:    Sun Jul 17 07:44:41 2016 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

root@ubuntu:/home/ubuntu# gdisk -l /dev/sdd
GPT fdisk (gdisk) version 0.8.8

Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present

Creating new GPT entries.
Disk /dev/sdd: 1565565872 sectors, 746.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 1F876634-0284-4A1C-8FDF-34A255B9DCCC
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1565565838
Partitions will be aligned on 2048-sector boundaries
Total free space is 1565565805 sectors (746.5 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
root@ubuntu:/home/ubuntu#

It is connected via a USB HDD dock. Perhaps this is the source of the problem. I'll rewire so it is plugged in directly to a SATA port on the motherboard. In the meantime here is the extra info requested.

root@ubuntu:/home/ubuntu# sg_readcap --16 /dev/sdd
READ CAPACITY (16) not supported
root@ubuntu:/home/ubuntu# sg_readcap  /dev/sdd
Read Capacity results:
   Last logical block address=1565565871 (0x5d50a3af), Number of blocks=1565565872
   Logical block length=512 bytes
Hence:
   Device size: 801569726464 bytes, 764436.5 MiB, 801.57 GB
root@ubuntu:/home/ubuntu# hdparm -N /dev/sdd

/dev/sdd:
 max sectors   = 5860533168/1(5860533168?), HPA setting seems invalid (buggy kernel device driver?)

Best Answer

root@ubuntu:/home/ubuntu# sg_readcap --16 /dev/sdd
READ CAPACITY (16) not supported

This means when your USB docking translate the capacity from the drive's ATA IDENTIFY DEVICE data (seen in hdparm -I / smartctl -i), it can at most report a size up to 32-bit (i.e. 0xffffffff, 4294967295) in terms of number of logical sectors. This is an inherit limitation of SCSI READ CAPACITY (10):

Logical Sector Size | Maximum capacity supported (TiB / TB)
         512        |              ~2.0 /  ~2.2
        4096        |             ~16.0 / ~17.6

Since your drive is an AF 512e drive that has totally 5860533168 / 0x15d50a3b0 512-byte logical sectors, which requires 33 bits to represent, only a SATA/USB bridge that supports SCSI READ CAPACITY (16) can handle it properly. When the size is truncated to 32-bit, it turns from:

101011101010100001010001110110000 (5860533168)

to

 01011101010100001010001110110000 (1565565872)

The Linux kernel, or probably all OSes, will basically never issue ATA IDENTIFY DEVICE command "directly" (i.e. encapsulated in a SCSI ATA PASS-THROUGH command) to USB drives, but SCSI READ CAPACITY commands (which you issued manually with sg_readcap), to get the capacity of them.

Only when the drives are actually SATA drive connected with a SATA/USB bridge, the command will be handled by the SCSI-ATA Translation Layer implemented in the bridge, which will then issue ATA IDENTIFY DEVICE command to the SATA drive to get the information it needs to form the response data for the READ CAPACITY command.

But utilities like hdparm and smartctl are (almost) exclusively for ATA drives, so they pretty much do everything with ATA PASS-THROUGH. (Also because they are userspace utilities, it is expected you, the user, will only use them on the appropriate type of devices.) That's why you end up getting different capacity in different places.

Related Question