Is It Safe to Take a Drive Image of the Current Working Drive? – Backup Guide

backupdddisk-image

I have to backup my hard disk. I want to use dd and put the image on an external hdd.

Can I do this using dd from the OS that resides on the hdd itself or do I have to boot from another device, for example a LiveCD?
Is it safe, in general, to take the image of a device, if the device is mounted and working?
What if the device is mounted, but I'm sure there's no other I/O operation, while dd is running?

I'm sure that rsync is the best tool to use for backups, specially the incremental ones.

But I'm interested in dd, because I want to backup up also other storage devices, and it copies also data stored on unpartitioned space. For example, my e-book reader use an unpartitioned space to store uboot, kernel and other data.

Best Answer

In general it is not safe. The FS assumes that operations are written in certain order so it can write new data of file and then make a pointer to it from other data, the exact details depend on filesystem. Imagine if following happens:

dd reads from location X which contains garbage or some data
Filesystem writes to location X
Filesystem writes to location X+1 pointer to location X
dd reads from location X+1 link to location X

From the point of view of backup you get a garbage data. However there are several ways to workaround it:

Freeze filesystem by filesystem specific command (I believe xfs_freeze is one and I don't know any other - but such option exists at least in theory)
Create a lvm snapshot and copy from it. The copy will be as-if you rebooted the computer (minus the HDD reordering) so it will be a dirty filesystem but the copy will be atomic. Note that some filesystems like XFS needs to be frozen first.
Use rsync as suggested by others. Now the copy is safe and you don't need LVM but the copy is not atomic. So while it avoids the above problem on filesystem level it might still run into problems with files (rather unlikely but one can imagine missing files while mv is executed in background for example)
Use filesystem with snapshoting such as btrfs, tux3, zfs, nilfs... Then you avoid both problems - you can just create a snapshot and copy from it by rsync having full atomicity. Note however that such filesystem often tend to be experimental.

As a last note - dd might not be a best way of backup. It copies a full disk which is often wasteful as you copy the 'garbage' as well. If you need to have a disk images something like partimage might be better. If you don't a better option is using either using a rsync, tar in differential/incremental mode etc. or a full backup system such as bacula, tarsnap or one of many others. Data deduplication may do wonders for the sizes of backups.

Washing the partition(s)

If the disk is in good working condition, you will get better compression if you wash the empty space on the disk with zeros. If the disk is failing, skip this step.

If you're imaging an entire disk then you will want to wash each of the partitions on the disk.

CAUTION: Be careful, you want to set the of to a file in the mounted partition, NOT THE PARTITION ITSELF!

mkdir image_source
sudo mount /dev/sda1 image_source
dd if=/dev/zero of=image_source/wash.tmp bs=4M
rm image_source/wash.tmp
sudo umount image_source

Making a Partition Image

mkdir image
sudo dd if=/dev/sda1 of=image/sda1_backup.img bs=4M

Where sda is the name of the device, and 1 is the partition number. Adjust accordingly for your system if you want to image a different device or partition.

Making a Whole Disk Image

mkdir image
sudo dd if=/dev/sda of=image/sda_backup.img bs=4M

Where sda is the name of the device. Adjust accordingly for your system if you want to image a different device.

Compression

Make a "squashfs" image that contains the full uncompressed image.

sudo apt-get install squashfs-tools
mksquashfs image squash.img

Streaming Compression

To avoid making a separate temporary file the full size of the disk, you can stream into a squashfs image.

mkdir empty-dir
mksquashfs empty-dir squash.img -p 'sda_backup.img f 444 root root dd if=/dev/sda bs=4M'

Mounting a compressed partition image

First mount the squashfs image, then mount the partition image stored in the mounted squashfs image.

mkdir squash_mount
sudo mount squash.img squash_mount

Now you have the compressed image mounted, mount the image itself (that is inside the squashfs image)

mkdir compressed_image
sudo mount squash_mount/sda1_backup.img compressed_image

Now your image is mounted under compressed_image.

EDIT: If you wanted to simply restore the disk image onto a partition at this point (instead of mounting it to browse/read the contents), just dd the image at squash_mount/sda1_backup.img onto the destination instead of doing mount.

Mounting a compressed full disk image

This requires you to use a package called kpartx. kpartx allows you to mount individual partitions in a full disk image.

sudo apt-get install kpartx

First, mount your squashed partition that contains the full disk image

mkdir compressed_image
sudo mount squash.img compressed_image

Now you need to create devices for each of the partitions in the full disk image:

sudo kpartx -a compressed_image/sda_backup.img

This will create devices for the partitions in the full disk image at /dev/mapper/loopNpP where N is the number assigned for the loopback device, and P is the partition number. For example: /dev/mapper/loop0p1.

Now you have a way to mount the individual partitions in the full disk image:

mkdir fulldisk_part1
sudo mount /dev/mapper/loop0p1 fulldisk_part1

Incremental backup of postgresql database

The best method to incremental backup Postgres is to use periodic hot physical backups and continuous WAL archiving.

http://www.postgresql.org/docs/9.1/static/continuous-archiving.html

The periodic physical backup can be done with rsync using the --copy-dest= option to take advantage of any previous copy reusing unchanged files.

This is a very effective way of backing up Postgres and can also be used to do Point In Time Recovery (PITR) in case of disaster.