Ubuntu – How to speed up a full disk dd

ddhard drivepartitioning

I am doing a dd on two identical drives with this command:

 dd if=/dev/sda of=/dev/sdb bs=4096

Both hard drives are the exact same model number, and both have 1TB of storage space. /dev/sda uses a blocksize of 4096. /dev/sda is a local drive and /dev/sdb is a remote caddy. I might be able to use the following protocols:

USB2.0 HighSpeed (Currently the plan)
Gigabit Over-The-Network clone (Really do not want to even try this)
USB3.0 (If I find my other drive caddy)
eSATA (If I find/buy a cable)
SATA (If I find/buy a cable, gotta love laptop CD drives)

Is there a way to run this drive copy that takes less that 96 hours? I am open to using tools other than dd.

I need to clone the following partitions (including UUIDs)

Fat32 EFI Partition (*)
NTFS Windows Partition (*)
HFS+ OSX Partition
EXT4 Ubuntu Partition (*)
Swap Partition (*)

* Supported by Clonezilla

I have tried Clonezilla (and it was MUCH faster), but it does not support HFS+ smart copying, which I need. Maybe the newest version supports this?

When I made my first clone, I did all of the partitions except HFS+ and it went very quickly. (No more than 3 hours total)

Best Answer

In my experience, I don't think there is something faster in the command line as dd. Adjusting the bs parameter can increase the speed, for example, I have 2 HDD that I know have a read/write speed greater than 100 MB/s so I do this:

dd if=/dev/sda of=/dev/sdb bs=100M

There is also pv (Needs to be installed first) that checks for the fastest speed on both drives and then proceeds on cloning. This has to be done of course from root:

pv < /dev/sda > /dev/sdb

With PV I got 156 MB/s

The nice thing about pv apart from the speed is that it shows the progress, current speed, time since it began and ETA. In regards to HFS+ I would not know, am just trying to help on the "speed" part. With pv or a very optimized bs parameter, you can do a 4 TB drive in less than 7 Hours (6 Hours 50 Minutes at a current speed of 150 MB/s).

enter image description here

I did a couple of tests with the connection types you were using and others I had available. I was using the Asus Z87 Pro and the Intel DZ68DP. This were my results, but first we need to know that the theoretical speeds for many transfer rates (Raw speeds) are just that, theory. Doing real tests revealed they are between 40% to 80% of that raw speed. This tests can change depending on Device used, connection type, motherboard, type of connecting cable, filesystem type and more. With that in mind, this is what I got (I only tested Write speed to the Device, read is typically higher):

Connected Device  -  Connection Type  -  Speed (Write Speed)
  USB 2.0                 USB 2.0              25 MB/s
  USB 3.0                 USB 2.0              35 MB/s
  USB 3.0                 USB 3.0              73 MB/s
  eSata                   eSata                80 MB/s
  Sata 2G HDD             Sata 2G              120 MB/s
  Sata 3G HDD             Sata 2G              140 MB/s
  Sata 3G HDD             Sata 3G              190 MB/s
  Sata 2G SDD             Sata 2G              170 MB/s
  Sata 3G SDD             Sata 2G              210 MB/s
  Sata 3G SDD             Sata 3G              550 MB/s

Related Solutions

Ubuntu – Clone internal HDD to new SSD

There is more than one way to accomplish getting your old system onto a new drive, but you didn't really ask it that way, you asked for how to clone the system.

I would just use gparted, myself, from the live CD so neither partition is mounted. You can shrink the original partition to the size you want it to be on the SSD, then copy and paste the partition to the new drive. If I remember correctly, this will reuse the same UUID, but you can change that on one or the other partition afterwards. The command for this is sudo tune2fs -U random /dev/sdb5 assigns UUID for sdb5.

If you don't want to change the old system, and if you want to keep it mounted for a while, you could change the UUID for the SSD partition, and edit your fstab. It's actually not hard at all, and is something you should learn about. It's pretty self-explanatory for someone with the knowledge you already seem to have. Once you assign a new UUID, you can see all of them with this command: sudo blkid -c /dev/null - the parameter -c specifies the cache file, and /dev/null means don't use a cache, so you always get any changes right away. I always use that form, and can see no downside unless you have a lot of partitions.

Once you get the UUID, you can copy and paste it over the old one in /etc/fstab using gedit or whatever text editor you prefer.

Personally, though, rather than taking time to resize the partition first, I'd simply copy the old install to the new disk. If you don't know how to install GRUB to the mbr, you might want to first install a base Ubuntu, then back up /etc/fstab, copy the old install over it, and then copy the fstab from the new install so it has only the correct entries.

Ubuntu – How to combine separate partitions created by initial install of kubuntu

Open the other partition in the file browser and press ctrl+L. You will be shown a path. Copy it. Then open a terminal and enter

cd "[whatever path]"

Where [whatever path] is the path you copied. Keep the quotes. They don't do any harm and make things easier if there are spaces or some other characters in the path. This will set your location in the current terminal to the mount point of your second partition.

Then enter these commands:

sudo chown -R [your user name] .
sudo chmod -R 777 .
sudo chown -R root "lost+found"
sudo chmod -R 700 "lost+found"

This will set the rights the way you want them to be. The first command makes you the owner of everything on the partition. The second command gives you all rights to it, but also every other user of the computer, in case you want to have several accounts and share files this way. If you don't want to do so, change 777 to 700 (others can't do anything) or to 722 (others can only read). The last 2 commands set everything back to the previous state for the (hidden) folder "lost+found" which is a special folder which you probably never will not have to use.

Best Answer

Related Solutions

Ubuntu – Clone internal HDD to new SSD

Ubuntu – How to combine separate partitions created by initial install of kubuntu

Related Question