Faster way to use dd to extract different parts of a binary file

ddfiles

I'm hoping someone may be able to help with the program dd which can do lots of things to files.

First of all, I use this program to cut a NES ROM into 2 files, graphics and code (1024 Kb for code, 512 Kb for graphics). It has a header at the start which is 16 bytes. And I have to remove that header first, before I put it into the "NESASM" assembler.

So that's where dd comes in. I have this setup which works.

dd if=MMC5 EXAtt MegaMan3.nes of=PRGrom.bin bs=16 skip=1 count=64k
dd if=MMC5 EXAtt MegaMan3.nes of=CHRgfx.bin bs=16 skip=65537 count=32k

It skips the first 16 bytes as I wanted. However, it's slow though, to complete. I've searched for MANY hours on google, seen tons of threads on stackexchange here, about using iflag=skip_bytes … to do this in bytes and not blocks. But the version of dd I have does not seem to support it.

I'm really hoping somebody here has a version of dd they can upload that has that "skip bytes" command supported. What I wanted to do was

  • skip the first 16 bytes only
  • and split it into 512 KB chunks so it would do this much faster than just 16 bytes at a time.

One method I tried was putting the bs=16 skip=1 at start of the input file line, then bs=512k on the output file line, but that doesn't work, it still only did it 16 bytes at a time. (I've tried tons of different ways like this)

I got dd from this page originally which does not have those "count bytes" or "skip bytes" options.

I don't want to go through the trouble of downloading Visual Studio 2010 or whatever is required to get the new dd version. Because everywhere I look, there's only source code files available to download on github and other places. I don't want all those GNU tools, just dd only.

I am on Windows XP, not Linux or other type of OS if that helps. I apologize if this is the wrong forum to ask. Most threads I have seen about dd were on this one specifically.

Best Answer

You could use GNU coreutils head instead:

{
  head -c 16   > /dev/null
  head -c 1m   > PRGrom.bin
  head -c 512k > CHRgfx.bin
} < 'MMC5 EXAtt MegaMan3.nes'

(that's Unix shell syntax, you may need to adapt if using a Microsoft command-line interpreter).

You may want to consider cygwin instead of that ancient gnuwin32 on Windows which will give you something closer to a proper modern GNU-like environment (Linux is not an OS btw, just the kernel found in a few Unix-like and non-Unix-like OSes).

skip_bytes was added to GNU dd in 2012 (coreutils 8.16). The latest version of that gnuwin32 is from 2005 (though I suppose that's contemporary to your Windows XP). You can find compiled versions of modern versions of GNU coreutils including dd in Cygwin and Git for Windows at least, both of which also come with POSIX shell interpreters. Cygwin has stopped supporting Windows XP circa 2016, but you should still be able to download versions from around that time from there for instance. Git for Windows dropped support for Windows XP after version 2.10.0, but it's still available online, and both postdate the coretutils 8.16 release.

Related Question