Ubuntu – Argument list too long

argumentscommand lineUbuntu

I'm trying to enter a command with VERY big argument (1 MB in size), but it says "Argument list too long." How can I enter that argument?

OS: Linux Ubuntu

Best Answer

There is no way to pass an argument between executables if it is larger than the kernel's ARG_MAX limit.

If you have a list of arguments which is too long, splitting it up into smaller pieces can be done e.g. with xargs. This runs the command as many times as necessary, supplying as many arguments as will fit.

The syntax is xargs command <file. If you omit command, it displays its arguments, like echo.

As a demo,

xargs -n 4 </etc/motd

will print the first four tokens on one line (first invocation), the next four through another, etc. (The -n argument sets a maximum number of arguments, so this doesn't use the ARG_MAX limit at all.)

If the command you want to run has side effects which are undesirable, this might not work. For example, if the command will overwrite any previously existing ./a.out file, you will obviously be left with the results from just the last run after it finishes.

If you can configure or modify command so it reads a file, or standard input, instead of a command-line argument, that will work around the restriction. A file or stream can be much larger than ARG_MAX, and often is.

Related Solutions

ls – Argument List Too Long for ls Command

Your error message argument list too long comes from the ***** of ls *.txt.

This limit is a safety for both binary programs and your Kernel. See ARG_MAX, maximum length of arguments for a new process for more information about it, and how it's used and computed.

There is no such limit on pipe size. So you can simply issue this command:

find -type f -name '*.txt'  | wc -l

NB: On modern Linux, weird characters in filenames (like newlines) will be escaped with tools like ls or find, but still displayed from *****. If you are on an old Unix, you'll need this command

find -type f -name '*.txt' -exec echo \;  | wc -l

NB2: I was wondering how one can create a file with a newline in its name. It's not that hard, once you know the trick:

touch "hello
world"

Makefile – Fix ‘Argument List Too Long’ Error

As @schily has already explained, this is not a shell problem, and cannot be worked around with xargs, quoting, splitting into more echo's with ;, etc. All the text from a make action is passed as argument/s to a single execve(2), and it can't be longer than the maximum size allowed by the operating system.

If you're using GNU make (the default on linux), you can use its file and foreach functions:

TEST = $(shell yes foobar | sed 200000q)

/tmp/junk:
        $(file >$@) $(foreach V,$(TEST),$(file >>$@,$V))
        @true

.PHONY: /tmp/junk

This will print all words from $(TEST) separated by newlines into the file named in $@. It's based on a similar example from make's manual.

Your Makefile could probably be reworked into something more manageable, that doesn't require fancy GNU features, but it's hard to tell how from the snippets you posted.

Update:

For the exact snippet from the question, something like this could do:

.hgignore : .hgignore_extra
    $(info Making $@)
    $(file >$@.new)
    $(file >>$@.new,# Automatically generated by Make. Edit .hgignore_extra instead.)
    $(shell tail -n 2 $< >>$@.new)
    $(file >>$@.new,)
    $(file >>$@.new,# The following files come from the Makefile.)
    $(file >>$@.new,syntax: glob)
    $(foreach L, $(IGNORE_DIRS) $(CLEAN_FILES) $(CLEAN_DIRS) $(REALCLEAN_FILES), $(file >>$@.new,$L))
    @mv -f $@.new $@
    @chmod a-w $@
.PHONY : .hgignore

I've changed it a little, so it first writes into .hgignore.new, and if everything goes well, only then move .hgignore.new to .hgignore. You'll have to change back the indenting spaces to tabs, because this dumb interface is mangling whitespaces.

Best Answer

Related Solutions

ls – Argument List Too Long for ls Command

Makefile – Fix ‘Argument List Too Long’ Error

Related Question