Repacking a tar or tar.gz file into smaller tar/tar.gz files

tar

I have a tar.gz file which contains a few million files in it. I want to divide the contents without decompressing it due to a cap on the number of files. Is there a way for me to divide a tar/tar.gz file in multiple tar/tar.gz.

if it is not possible, can it be done in python instead?

1st post let me know if I did something wrong.

Edit: I don't want to put it back together. I want to be able to work on each 'splitted' files.

Best Answer

You can split it with split, but the individual pieces won't make any sense until put back together. If you want to split it into self-contained archives, you will need to decompress it.

Related Solutions

Tar extract into directory with same base name

With gnu tar, you could use --xform (or --transform) to prepend /prefix/ to each file name:

tar -xf myArchive.tar.gz --xform='s|^|myArchive/|S'

note there's no leading / in prefix/ and the sed expression ends with S to exclude symbolic link targets from file name transformations.
To test it (dry-run):

tar -tf myArchive.tar.gz --xform='s|^|myArchive/|S' --verbose --show-transformed-names

To get you started, here's a very simplistic script that you could invoke as extract <file>:

STRIP=${1%.*}                                #strip last suffix
NAME=${STRIP%.tar}                           #strip .tar suffix, if present
tar -xf "$1" --xform="s|^|$NAME/|S"          #run command

Scp changing permissions on /tmp directory

You've included . in your source collection given to tar. The problem occurs when you extract the tarball and the permissions are applied. Your source directory . is extracted and permissions applied. Unfortunately at the point of extraction . is actually /tmp.

The fix is either to exclude . from the tarball or to create a temporary directory within /tmp in which to extract it all.

Best Answer

Related Solutions

Tar extract into directory with same base name

Scp changing permissions on /tmp directory

Related Question