Portable way to invoke tar with a list of files from stdin

busyboxio-redirectionoptionsstdintar

I want to create a .tgz from the content of a directory. I also want to strip the leading "./" from the tar'ed content.
I had done this as follows:

cd /path/to/files/ && find . -type f | cut -c 3- | xargs czf /path/to/tgz/myTgz.tgz

I learned recently that using xargs may not be the best way to pull this off because xargs may invoke tar multiple times if the cmdline arg list gets too long, and I was advised to make use of tar's ability to read a list of input files from stdin. I ended up finding this article on how to do this. However, I find that the recommendation…

cd /path/to/files/ && find . -type f | cut -c 3- | tar czf foo.tgz -T -

…seems to not be portable. It runs fine on my dev PC, but on a busybox target, I get the following error from running the same command:

tar: can't open '-': No such file or directory

So, my question: is there a truly portable/global way to invoke tar to create a .tgz by feeding it input files from stdin (as opposed to cmdline arguments)?

(It is not an option available to me to install alternatives to tar such as gnutar/bsdtar/etc.)

(Secondary question: Why does the "-T -" argument to tar denote "read files from stdin"? From the tar man page, all I could find was that "-T" means:

get names to extract or create from FILE

… but I couldn't see any reference to a plain "-")

Best Answer

It is a common convention to interpret - to mean standard input where an input file name is expected, and to mean standard output where an output file name is expected. Because this is a common convention, the short help summary in the GNU tar man page does not mention it, but the complete manual (usually available locally through info tar) does. The POSIX command line utility syntax guidelines includes this convention, so it's pretty widespread (but it's always a choice on the part of the author of the program).

BusyBox utilities do follow this convention. But the manual does not mention tar as supporting the option -T, and neither does the version on the machine I'm posting this (1.27.2 on Ubuntu). I don't know why you're getting the error “: No such file or directory” rather than “invalid option -- 'T'”. It seems that your tar interprets -T as an option that does not take an argument, then sees - as a file name. Since in this context tar needs a file name to put in the archive, and not just some content that comes from a file, it would not make sense to use the stdin/stdout interpretation for -.

BusyBox utilities support a restricted set of functionality by design, because they're intended for embedded systems where the fancier features of GNU utilities wouldn't fit. Apparently -T is not a feature that the BusyBox designers considered useful.

I don't think BusyBox tar has any way to read file names from stdin. If you need to archive a subset of the files in a directory and you don't need any symbolic links in the archive, a workaround is to create a forest of symbolic links in a temporary directory and archive this temporary directory.

It's not clear exactly why you're using find. If you only want the files in the current directory, why not tar czf /path/to/archive.tgz -- * ? Your command does make sense if there are subdirectories and you want to archive the files in these subdirectories, but not the directories themselves (presumably to restore them in a place where the directory structure must exist but may have different permissions). In this case a leading ./ wouldn't do any harm.

Related Question