Normally if I want to copy a raw partition /dev/sdaX
(with i/o errors when reading), I would just go with ddrescue <src> <dst>
.
The case here is that I'd like to be able to copy this partition in smaller chunks (e.g. 1GB partition in 10*100MB chunks), but not all at once, and glue them together afterwards. Imagine this as a scenario where you want to copy that 1GB partition from PC_A
to PC_B
but you only have a 100MB usb stick available.
Is this technically possible? Would ddrescue
be able to do that, if so, how?
Best Answer
This is technically possible. The filesystem on the USB stick must support sparse files for my method to work.
To tell
ddrescue
to recover a part of a file, use-m
:(source)
You need to prepare many domain-mapfiles, each describing one rescue domain, one chunk. Altogether they should cover the entire device. Read about the mapfile structure. The 0th domain-mapfile
chunk00.map
will be:This means the chunk starts at offset 0 and takes 104857600 bytes. The latter number is interpreted as decimal.
ddrescue
creates mapfiles with hexadecimal values (0x…
) but it understands decimal. A leading zero denotes octal, so don't use leading zeros.Alternatively you can specify these numbers with
-i
and-s
command line options; but because we will need the exact same numbers on PC_A and PC_B, it's better to have them in a file you can copy. If you use the right values on PC_A and copy the file to PC_B then you will use the right values on PC_B. Typing commands with-i
and-s
on both systems independently is more error-prone.104857600 bytes is 100 mebibytes. I did not use 100000000 (100 megabytes) because we're going to use either
-b 512
or-b 4096
(depending on the physical sector size of the source device) and the chunk should take an integer number of sectors. 104857600 is a multiple of 4096 (and therefore a multiple of 512), 100000000 is not.chunk01.map
will be:chunk02.map
:and so on. The starting position for the chunk N+1 is the starting position of the chunk N plus the size of the chunk N. In general the chunks may overlap but we don't want them to overlap.
Read the 0th chunk:
You need to do this for each chunk, you need to generate
chunk01.raw
fromchunk01.map
,chunk02.raw
fromchunk02.map
and so on.common.map
should be the same mapfile each time, each invocation ofddrescue
on PC_A should update the common mapfile.Resulting chunks with non-zero offset (i.e. all chunks but
chunk00.raw
) will be sparse.If the filesystem on the USB stick did not support sparse files then each chunk would be bigger than the previous one due to explicit zeros in the beginning. The last chunk would be as big as
/dev/sdaX
, this defies the purpose. A workaround can be to write a chunk to a local filesystem that supports sparse files, pack it (with or without compression) somehow and finally unpack on PC_B as sparse. It's way easier if the filesystem on the USB stick supports sparse files.After reading each chunk you should transfer the chunk to PC_B. You need to transfer
chunk*.raw
and its correspondingchunk*.map
. If you need to copy a sparse chunk to another filesystem, usecp --sparse=always …
. You don't necessarily need to copy though; building the final image can be done by reading directly from the USB stick.Before transferring the 0th chunk, make sure the final image
final.raw
on PC_B is empty or nonexistent. Transfer the 0th chunk from the USB:Do this with all chunks as they come, writing to the same
final.raw
; just remember to use the rightchunk*.map
each time. Hereddrescue
reads regular files, we don't expect read errors, there's no need for a mapfile.After transferring all the chunks this way,
final.raw
on PC_B should be complete. The filecommon.map
on PC_A is the corresponding mapfile. It's as if you didon PC_A, except
final.raw
is now on PC_B.Notes:
In case of read errors on PC_A, a chunk may be smaller than you expect. See "conclusions" in this answer of mine (where "chunk" means "one or more sectors", "a fragment", it's different than our chunk):
If our last chunk is smaller then
final.raw
on PC_B will be smaller than/dev/sdaX
on PC_A. It would be exactly like this if you letddrescue
write tofinal.raw
on PC_A in one run.In my tests a chunk that starts within
/dev/sdaX
and reaches beyond the end ofsdaX
generates adequately smallerchunk*.raw
file. A chunk that starts beyond the end ofsdaX
makesddrescue
on PC_A complain and fail. This means you don't need to pay attention to the size ofsdaX
, you don't need to define the last chunk accurately. While buildingchunk*.map
files you can increase the starting position by your desired size each time, until somechunk*.raw
is created smaller because of the end ofsdaX
. Then you try toddrescue
the next chunk, getCan't start reading at pos … . Input file is only … bytes long.
from the tool and this way you find out the wholesdX
has been read.common.map
should confirm this.