I want to write a CGI, which must read a specified number of bytes from STDIN. My idea is to do it this way:
dd bs=$CONTENT_LENGTH count=1
But I was wondering, if the block size is limited by anything else but the RAM.
$ dd bs=1000000000000
dd: memory exhausted by input buffer of size 1000000000000 bytes (931 GiB)
The manual page of GNU's coreutils does not specify any limit.
Best Answer
The POSIX specifications for
dd
don’t specify a maximum explicitly, but there are some limits:size_t
, since that’s the type of the number of bytes to read given to theread
function;read
is also specified to have a limit ofSSIZE_MAX
;read
only transfers up to 2,147,479,552 bytes anyway.On a 64-bit platform,
size_t
is 64 bits in length; in addition, it’s unsigned, sodd
will fail when given values greater than 264 – 1:On Linux on 64-bit x86,
SSIZE_MAX
is 0x7fffffffffffffffL (runecho SSIZE_MAX | gcc -include limits.h -E -
to check), and that’s the input limit:Once you find a value which is accepted, the next limit is the amount of memory which can be allocated, since
dd
needs to allocate a buffer before it can read into it.Once you find a value which can be allocated, you’ll hit the
read
limit (on Linux and other systems with similar limits), unless you use GNUdd
and specifyiflag=fullblock
:(
dd
copied just under 231 bytes, i.e. the Linux limit mentioned above, not even half of what I asked for).As explained in the Q&A linked above, you’ll need
fullblock
to reliably copy all the input data in any case, for any value ofbs
greater than 1.