Best Compression Method – Top Techniques for Data Compression

compression

I want to compress a 16GB folder, but what's the best method? tar.gz? tar.bz2 rar? 7z? Would the archive be smaller if I first compressed in a method, then copy the compressed archive to a new folder, then re-compressed in some other method? I need to make it fit on a DVD (output maybe 8.5GB, don't remember) but putting "4370 MB" makes the compressed file be 2.5GB part.

BTW, what's the default compression method on Ubuntu?

Best Answer

The default is gz. The best results I get with 7z though.

Here is the results for a 1.4 Gb virtualbox container:

enter image description here

Best compression – size in MB:

7z 493
rar 523
bz2 592
lzh 607
gz 614
Z 614
zip 614
.arj 615
lzo 737
zoo 890

Source

enter image description here

Install

 sudo apt-get install p7zip-full

Related Solutions

Ubuntu – Multi-Core Compression tools

There are two main tools. lbzip2 and pbzip2. They're essentially different implementations of bzip2 compressors. I've compared them (the output is a tidied up version but you should be able to run the commands)

cd /dev/shm  # we do all of this in RAM!
dd if=/dev/urandom of=bigfile bs=1024 count=102400

$ lbzip2 -zk bigfile 
Time: 0m3.596s
Size: 105335428 

$ pbzip2 -zk bigfile
Time: 0m5.738s6
Size: 10532460

lbzip2 appears to be the winner on random data. It's slightly less compressed but much quicker. YMMV.

Maximal Compressed Screenshots – How to Get Them

optipng is certainly worth a look at . It's command-line but you could write a .desktop handler for it. It's fairly simple to use for batching:

optipng -o7 *.png

But note it's destructive. It will overwrite your files. This is usually fine but if you're using a fat format (Adobe Fireworks PNG for example), you might lose useful. Here is the full list of options:

$ optipng -h
OptiPNG 0.6.4: Advanced PNG optimizer.
Copyright (C) 2001-2010 Cosmin Truta.

Synopsis:
    optipng [options] files ...
Files:
    Image files of type: PNG, BMP, GIF, PNM or TIFF
Basic options:
    -?, -h, -help   show this help
    -o <level>      optimization level (0-7)        default 2
    -v          verbose mode / show copyright and version info
General options:
    -fix        enable error recovery
    -force      enforce writing of a new output file
    -keep       keep a backup of the modified files
    -preserve       preserve file attributes if possible
    -quiet      quiet mode
    -simulate       simulation mode
    -snip       cut one image out of multi-image or animation files
    -out <file>     write output file to <file>
    -dir <directory>    write output file(s) to <directory>
    -log <file>     log messages to <file>
    --          stop option switch parsing
Optimization options:
    -f  <filters>   PNG delta filters (0-5)         default 0,5
    -i  <type>      PNG interlace type (0-1)        default <input>
    -zc <levels>    zlib compression levels (1-9)       default 9
    -zm <levels>    zlib memory levels (1-9)        default 8
    -zs <strategies>    zlib compression strategies (0-3)   default 0-3
    -zw <window size>   zlib window size (32k,16k,8k,4k,2k,1k,512,256)
    -full       produce a full report on IDAT (might reduce speed)
    -nb         no bit depth reduction
    -nc         no color type reduction
    -np         no palette reduction
    -nx         no reductions
    -nz         no IDAT recoding
Optimization details:
    The optimization level presets
        -o0  <=>  -o1 -nx -nz
        -o1  <=>  [use the libpng heuristics]   (1 trial)
        -o2  <=>  -zc9 -zm8 -zs0-3 -f0,5    (8 trials)
        -o3  <=>  -zc9 -zm8-9 -zs0-3 -f0,5  (16 trials)
        -o4  <=>  -zc9 -zm8 -zs0-3 -f0-5    (24 trials)
        -o5  <=>  -zc9 -zm8-9 -zs0-3 -f0-5  (48 trials)
        -o6  <=>  -zc1-9 -zm8 -zs0-3 -f0-5  (120 trials)
        -o7  <=>  -zc1-9 -zm8-9 -zs0-3 -f0-5    (240 trials)
    The libpng heuristics
        -o1  <=>  -zc9 -zm8 -zs0 -f0        (if PLTE is present)
        -o1  <=>  -zc9 -zm8 -zs1 -f5        (if PLTE is not present)
    The most exhaustive search (not generally recommended)
      [no preset] -zc1-9 -zm1-9 -zs0-3 -f0-5    (1080 trials)
Examples:
    optipng file.png                (default speed)
    optipng -o5 file.png            (moderately slow)
    optipng -o7 file.png            (very slow)
    optipng -i1 -o7 -v -full -sim experiment.png

There are other tools (pngcrush for example) and you might find they're more useful but I've never strayed from optipng, just for its no-nonsense simlpicity.

There's a fairly comprehensive PNG compression shoot-out here that's quite fun:

http://css-ig.net/png-tools-overview

Best Answer

Related Solutions

Ubuntu – Multi-Core Compression tools

Maximal Compressed Screenshots – How to Get Them

Related Question