Software Recommendations – Best Tools for Compression of Large Files

compressionsoftware-rec

I am trying to compress large datasets (from 15-400 GB each).
I am looking for the best compression software for Mac that has

  1. High compression rate.
  2. Is fast

I have tried 7zX, which has excellent compression rate on Ultra settings, but is very slow.
I wonder if I can change the settings so it

  1. uses all cores I have
  2. the max RAM
  3. and max CPU.

So I can shorten that time. Or if there is another software that offers this option.
Of, course if anybody can recommend any better software, I'd be happy to here from it.

Also about workflows on how to deal with big files that need to be compressed/decompressed later.

Thanks.

Best Answer

There are versions of bzip (pigz) and bzip2 (pbzip2) which are multi-threaded. Both will use 100% of all cores.

Using the maximum RAM doesn't make sense and the algorithms will take as much as necessary to run, not more.

Which algorithm is best, depends on your use case and data, but pigz will probably be one of the fastest and 7z one of the slowest, but with better compression ratio.

Related Question