CP Command – Max Source Files Number Arguments for Copy Utility

argumentscp

Consider that there are countless number of files under /src/

cp /src/* /dst/

How many files cp will successfully process?

Best Answer

That depends greatly on the system and version, on the number and size of the arguments and on the number and size of environment variable names.

Traditionally on Unix, the limit (as reported by getconf ARG_MAX) was more or less on the cumulative size of:

  • The length of the argument strings (including the terminating '\0')
  • The length of the array of pointers to those strings, so typically 8 bytes per argument on a 64bit system
  • The length of the environment strings (including the terminating '\0'), an environment string being by convention something like var=value.
  • The length of the array of pointers to those strings, so typically 8 bytes per argument on a 64bit system

Bearing in mind that cp also counts as an argument (is the first argument).

On Linux, it depends on the version. The behaviour there changed recently where it's not longer a fixed space.

Checking on Linux 3.11, getconf ARG_MAX now reports a quarter of the limit set on the stack size, or 128kiB if that's less than 512kiB).

(zsh syntax below):

$ limit stacksize
stacksize       8MB
$ getconf ARG_MAX
2097152
$ limit stacksize 4M
$ getconf ARG_MAX
1048576

That limit is on the cumulative size of the argument and environment strings and some overhead (I suspect due to alignment consideration on page boundaries). The size of the pointers is not taken into account.

Searching for the limit, I get:

$ /bin/true {1..164686}
$ /bin/true {1..164687}
zsh: argument list too long: /bin/true
$ x= /bin/true {1..164686}
$ x=1 /bin/true {1..164686}
zsh: argument list too long: /bin/true

The maximum cumulative size before breaking in that case is:

$ (env _=/bin/true x=;print -l /bin/true {1..164686}) | wc -c
1044462

Now, that does not mean that you can pass 1 million empty arguments. On a 64 bit system, 1 million empty arguments make a pointer list of 8MB, which would be above my stack size of 4MiB.

$ IFS=:; /bin/true ${=${(l.1000000..:.)${:-}}}
zsh: killed     /bin/true ${=${(l.1000000..:.)${:-}}}

(you'll noticed it's not a E2BIG error. I'm not sure at which point the process gets killed there though if it's within the execve system call or later).

Also note (still on Linux 3.11) that the maximum size of a single argument or environment string is 128kiB, regardless of the size the stack.

$ /bin/true ${(l.131071..a.)${:-}} # 131072 OK
$ /bin/true ${(l.131072..a.)${:-}} # 131073 not
zsh: argument list too long: /bin/true
$ /bin/true ${(l.131071..a.)${:-}} ${(l.131071..a.)${:-}} # 2x 131072 OK
Related Question