dd – Enlarge Each Input Block to Double Size

I found a few examples that claimed this but I could not manage to get dd to convert a file with a given block size to double that block size.

dd if=disk256bytesectors.img of=disk512bytesectors.img cbs=256 ibs=512 obs=512 conv=sync

My disk img is 10 megabytes in size I was expecting a new image of 20 megabytes, but no.

So what I need to happen is each 256 byte block get converted to a 512 byte block with the second half of each 512 byte block being null, zero, space or anything.

The dd manual state conv=sync

pads any input block shorter than ibs to that size with null bytes before conversion and output.

But the input block will never be shorter than itself?!?! So what does that mean?

Anyway how can I do this?

Best Answer

You might be able to use socat which uses a Unix domain socket when you specify exec: for a command. This can be made of type datagram (type=2) to ensure the input to dd is only 256 bytes.

For example, for 2 bytes per datagram (-b2) padded to 4 (bs=4):

$ echo abcdefx | socat -u -b2 - exec:'dd bs=4 conv=sync',type=2 | od -c   
0000000   a   b  \0  \0   c   d  \0  \0   e   f  \0  \0   x  \n  \0  \0

Note, the above use of a pipe for stdin to socat is just for this test. Normally, you should provide the input file directly on stdin so that socat has no trouble reading whole "blocks" of the wanted -b size (i.e. <file socat -u ...).

Related Solutions

Dd, seek with one block size, write with another block size

Solution:

dd if='input_file.bin'           \
   of='/dev/sd{X}'               \
   bs={desired write block size} \
   seek={start offset in bytes}  \
   count={write size in bytes}   \
   oflag=seek_bytes              \
   iflag=count_bytes

From the man page:

count_bytes
    treat 'count=N' as a byte count (iflag only)

...

seek_bytes
    treat 'seek=N' as a byte count (oflag only)

This does seem to slow down the transfer a bit, but at least puts it in MB/s, instead of kB/s. Also, be sure to check the man page on your system, as it seems the ones available on the web (i.e. googling 'man dd') don't include these options.

Linux – Why specify block size when copying devices of a finite size

When is dd suitable for copying data? (or, when are read() and write() partial) points out an important caveat when using count: dd can copy partial blocks, so when given count it will stop after the given number of blocks, even if some of the blocks were incomplete. You may therefore end up with fewer than bs * count bytes copied, unless you specify iflag=fullblock.

The default block size for dd is 512 bytes. count is a limit; as your question hints it isn't required when copying a device of finite size, and is really intended to copy only part of a device.

I think there are two aspects to consider here: performance and data recovery.

As far as performance is concerned, you ideally want the block size to be at least equal to, and a multiple of, the underlying physical block size (hence 2048 bytes when reading a CD-ROM). In fact nowadays you may as well specify larger block sizes to give the underlying caching systems a chance to buffer things for you. But increasing the block size means dd has to use that much more memory, and it could be counter-productive if you're copying over a network because of packet fragmentation.

As far as data recovery is concerned, you may retrieve more data from a failing hard disk if you use smaller block sizes; this is what programs such as dd-rescue do automatically: they read large blocks initially, but if a block fails they re-read it with smaller block sizes. dd won't do this, it will just fail the whole block.

Best Answer

Related Solutions

Dd, seek with one block size, write with another block size

Linux – Why specify block size when copying devices of a finite size

Related Question