Security – Fill Device with Zeros Without Overwriting Existing Zeros

ddSecurity

I have a USB flash drive usb 3, the reading speed is much more than the writing speed.
Let's say that 99% of the flash memory is already full with zeros, and I would like to fill it with zeros until 100%, by overwriting all the memory flash with zeros dd if=/dev/zero of=/dev/FLASH.

This process is going to be long, and it will minimize the life expectancy of the flash drive.

I thought, maybe it would be much quicker to check which areas are non-zero, and overwrite only those non-zertos areas with zeros?

Are there anyways of doing this? If it is interesting, I would need all this for security reasons.

Best Answer

Security reasons aside, let's do it. We can (ab)use GNU ddrescue.

To detect sectors of zeros --generate-mode is useful.

When ddrescue is invoked with the --generate-mode option it operates in "generate mode", which is different from the default "rescue mode". That is, if you use the --generate-mode option, ddrescue does not rescue anything. It only tries to generate a mapfile for later use.

[…]

ddrescue can in some cases generate an approximate mapfile, from infile and the (partial) copy in outfile, that is almost as good as an exact mapfile. It makes this by simply assuming that sectors containing all zeros were not rescued.

[…]
ddrescue --generate-mode infile outfile mapfile

^(source)

Let's pretend your device is outfile from previous ddrescue run. We cannot use it as infile (because ddrescue refuses to work when infile and outfile are the same file), we need a dummy one, /dev/zero will do. We should know the physical sector size of your device and use it with -b option. This command may help:

lsblk -o NAME,PHY-SeC /dev/FLASH

Here I assume it's 512.

ddrescue -b 512 --generate-mode /dev/zero /dev/FLASH flash.map

Now flash.map describes every sector either as non-tried (?) or as finished (+), depending on whether it was full of zeros or not. The next step is to fill non-zero sectors with zeros; --fill-mode is perfect for this job:

When ddrescue is invoked with the --fill-mode option it operates in "fill mode", which is different from the default "rescue mode". That is, if you use the --fill-mode option, ddrescue does not rescue anything. It only fills with data read from infile the blocks of outfile whose status character from mapfile coincides with one of the type characters specified as argument to the --fill-mode option.

^(source)

We must use the same -b value as with --generate-mode, additionally --force to overwrite the output device. This is the command:

ddrescue -b 512 --force --fill-mode=+ /dev/zero /dev/FLASH flash.map

This time /dev/zero is not just a dummy argument, it's the actual source of data (zeros) written to the device.

After ddrescue finishes, invoke sync. Now /dev/FLASH is filled with zeros.

Related Solutions

The fastest way to write all sectors on a drive

If your goal is to really "zero" the drive, then I bet the fastest you can get is to issue a low-level secure erase command using hdparm (see here for step-by-step instructions).

Note two things:

As hdparm manpage vividly states, the operation is "DANGEROUS".
On the other hand, it may also repair bad blocks.

Hard-Disk – How to Ignore Write Errors While Zeroing a Disk

I prefer badblocks in destructive write mode for this. It writes, it continues doing so when it hits errors, and finally it tells you where those errors were, and this information may help you decide what to do next (Will It Blend?).

# badblocks -v -b 4096 -t random -o badblocks.txt -w /dev/destroyme
Checking for bad blocks in read-write mode
From block 0 to 2097151
Testing with random pattern: done
Reading and comparing: done
Pass completed, 52105 bad blocks found. (0/52105/0 errors)

And the block list:

# head badblocks.txt
2097000
2097001
2097002
2097003
2097004

And what's left on the disk afterwards:

# hexdump -C /dev/destroyme
00000000  be e9 2e a5 87 1d 9e 61  e5 3c 98 7e b6 96 c6 ed  |.......a.<.~....|
00000010  2c fe db 06 bf 10 d0 c3  52 52 b8 a1 55 62 6c 13  |,.......RR..Ubl.|
00000020  4b 9a b8 d3 b7 57 34 9c  93 cc 1a 49 62 e0 36 8e  |K....W4....Ib.6.|

Note it's not really random data - the pattern is repetitive, so if you skipped 1MiB you'd see the same output again.

It will also try to verify by reading the data back in, so if you have a disk that claims to be writing successfully but returns wrong data on readback, it will find those errors too. (Make sure no other processes write to the disk while badblocks is running to avoid false positives.)

Of course with a badly broken disk this may take too long: there is no code that would make it skip over defective areas entirely. The only way you could achieve that with badblocks would be using a much larger blocksize.

I'm not sure if ddrescue does this any better; it's supposed to do that in the other direction (recover as much data as fast as possible). You can do it manually for dd/ddrescue/badblocks by specifying first/last block...

Best Answer

Related Solutions

The fastest way to write all sectors on a drive

Hard-Disk – How to Ignore Write Errors While Zeroing a Disk

Related Question