GNU tar formats suggests tar -c --format=posix
or tar -c --format=pax
is the only way to create a portable tar file containing path names longer than 255 characters.
The Solaris tar man page doesn't mention supporting that format, but the Solaris pax man page does.
So first, I'd try to create the archive on your Cygwin (or Linux) system using tar -c --format=posix -f dir.tar dir
, and to extract it on your Solaris system using pax -r -x pax -f dir.tar
, or likely just pax -r -f dir.tar
.
Alternatively, if you have cpio
installed on both systems, it seems like the crc
format supports 1,023 character paths on both systems. That would look something like find -L dir | cpio -o -H crc > dir.cpio
to create and cpio -i -d < dir.cpio
to extract it.
Finally, you could try adding the -i
flag to ignore directory checksum errors when extracting on Solaris, e.g. tar -x -i -f dir.tar
. Checksumming problems says this can be needed if any of the file names in the tar archive are non-ASCII.
Other reading:
I wrote this bash script to do it.
It basically forms an array containing the names of the files to go into each tar, then starts tar
in parallel on all of them.
It might not be the most efficient way, but it will get the job done as you want.
I can expect it to consume large amounts of memory though.
You will need to adjust the options in the start of the script.
You might also want to change the tar options cvjf
in the last line (like removing the verbose output v
for performance or changing compression j
to z
, etc ...).
Script
#!/bin/bash
# User configuratoin
#===================
files=(*.log) # Set the file pattern to be used, e.g. (*.txt) or (*)
num_files_per_tar=5 # Number of files per tar
num_procs=4 # Number of tar processes to start
tar_file_dir='/tmp' # Tar files dir
tar_file_name_prefix='tar' # prefix for tar file names
tar_file_name="$tar_file_dir/$tar_file_name_prefix"
# Main algorithm
#===============
num_tars=$((${#files[@]}/num_files_per_tar)) # the number of tar files to create
tar_files=() # will hold the names of files for each tar
tar_start=0 # gets update where each tar starts
# Loop over the files adding their names to be tared
for i in `seq 0 $((num_tars-1))`
do
tar_files[$i]="$tar_file_name$i.tar.bz2 ${files[@]:tar_start:num_files_per_tar}"
tar_start=$((tar_start+num_files_per_tar))
done
# Start tar in parallel for each of the strings we just constructed
printf '%s\n' "${tar_files[@]}" | xargs -n$((num_files_per_tar+1)) -P$num_procs tar cjvf
Explanation
First, all the file names that match the selected pattern are stored in the array files
. Next, the for loop slices this array and forms strings from the slices. The number of the slices is equal to the number of the desired tarballs. The resulting strings are stored in the array tar_files
. The for loop also adds the name of the resulting tarball to the beginning of each string. The elements of tar_files
take the following form (assuming 5 files/tarball):
tar_files[0]="tar0.tar.bz2 file1 file2 file3 file4 file5"
tar_files[1]="tar1.tar.bz2 file6 file7 file8 file9 file10"
...
The last line of the script, xargs
is used to start multiple tar
processes (up to the maximum specified number) where each one will process one element of tar_files
array in parallel.
Test
List of files:
$ls
a c e g i k m n p r t
b d f h j l o q s
Generated Tarballs:
$ls /tmp/tar*
tar0.tar.bz2 tar1.tar.bz2 tar2.tar.bz2 tar3.tar.bz2
Best Answer
tar can create or extract tar's but it can't operate on streams in this way.
You'll need something like
tar-stream
- it even has an example that does what you are asking:https://github.com/mafintosh/tar-stream#modifying-existing-tarballs