Linux – How to refresh the magnetic state on a disks with backups

backupdddisklinuxstorage

I have a couple of large disks with backup/archive material on them. They're ext4. Regarding the ones of those that will be stored for a couple of years without reading the whole disc again I've been thinking of a way to refresh the disks magnetic state. Shelf life of drives seems to be a matter of debate everywhere I've been looking for an answer, but it seems after a couple of years (say 5 or so) of storage it would be wise to refresh the data in some way (?)

I've seen this suggested:

dd if=/dev/sda of=/dev/sda

Is it safe? Is it useful?

What I'm looking to do is another thing than a fsck or a dd if=/dev/sda of=/dev/null, both of which will probably discover existing magnetic drop outs on the disk.

What I want to do is to refresh the magnetic data before the magnetic charges on the disk lowers below a readable level. How can I do this?

Best Answer

Generally you can't really refresh the whole disk without reading/writing all of it. fsck is unlikely to provide what you need - it works with the file system not the underlying device hence it mostly just scans file system meta data (inodes and other file system structures).

badblocks -n might be an option to dd if=X of=X. In any case you probably want to use large blocks to speed things up (for dd something like bs=16M, for badblocks this would read -b 16777216, or -b $((1<<24)) in reasonable shells). You'll probably also want to use conv=fsync with dd.

As for the safety of dd with the same input and output device - it reads block from input and writes it to output, so it should be safe (I have re-encrypted an encrypted partition like this on several occasions, by creating loop devices with the same underlying device and different passwords and then dd'ing from one to the other) - at least for some types of physical media: for example with shingled drives it is definitely not obvious to me, that it is 100% failure-proof.

Related Solutions

Debian – I can see the LVM but I can’t initialize it. Missing UUID of one of the disks

You screwed up. You were told you couldn't format the disk because it was in use. It was in use. You were trying to format one of the existing disks, not the new one. Now you formatted the existing drive and lost your data. You will need to restore from backup.

You can see from the pvdisplay output that /dev/sdd1 is 100% free, so that seems to be the new drive.

Linux – Disable writeback cache throttling – tuning vm.dirty_ratio

The problem is not the dirty-throttling, but the immediate syncing. You can check it with the bo column in vmstat 1 - I get:

   -----io----
   bi    bo
    0     0
    0     0
    0  4000
    0     0
    ...

after dd if=/dev/zero of=16/test bs=4k count=1000. (The vmstat output is just 1K-"units" - /proc/diskstats has 512B-units)

Adding conv=notrunc avoids that, as does a rm before. See sourcejedi's comment.

The ext4 mount option noauto_da_alloc exactly is meant for that: no extra sync for these "write-file-anew" operations. man ext4 has some good examples. The question goes back to what should "write" or "save" file mean in an editor, and how to make that (power-) fail-safe.

You can also check the other side: the dirty pages growing:

]# grep nr_dirty /proc/vmstat      
nr_dirty 7886
nr_dirty_threshold 427666
nr_dirty_background_threshold 170815

The thresholds are the ratios applied to the current available memory. Only since I quite disabled the periodic writebacks I see more than a dozen dirty pages.

vim

It is really a complex situation; vim has at least two crucial options: fs and wb: filesync after write and overwrite via backup.

On ext2 it is enough to set nofs in vim. On ext4 (with default delayed allocation) the noauto_da_alloc mount option is also needed to prevent an immediate syncing, this time by the filesystem.

(with dd you have the option to (over)write only a part of a file, unlike cp or vim)

Best Answer

Related Solutions

Debian – I can see the LVM but I can’t initialize it. Missing UUID of one of the disks

Linux – Disable writeback cache throttling – tuning vm.dirty_ratio

vim

Related Question