I have a Hex-core Xeon X5650 running Windows 7 x64.
I'm currently doing simultaneous compression of three NTFS hard drives, 2TB each, with the commands:
compact e: /i /c /s
compact f: /i /c /s
compact g: /i /c /s
Its getting reasonable compression: up to 40:1, and typically 2:1.
The resulting CPU load is only about 3%
Does anyone know why is the CPU load so low?
Does anyone know if the Win7 realtime compression/decompression kernel is using SSE3 or SSSE3 or SSE4.2 instructions behind the scenes?
Best Answer
I would expect that your cpu usage is so low because a number of factors but firstly keep in mind that the compression algorithm is designed for speed rather than high compression rates.
The fundamental problem here is whether you are compressing thousands upon thousands of small files or several GB of humongous files.
If you are compressing files under 2MB I would expect that most of the time taken to compress the files is take up in the operations needed to find a file in the filesystem, get the hard drive to give you the data, and then writing the compressed data back, then repeat with the next tiny file. The compression algorithm can probably handle around 40-50MB/s on a modern machine (I would expect but am not certain of this) and so the hard drives seek and data transfer times are truly the limiting factors for those files, compression times are likely to be nearly nil.
For larger files you should almost certainly be seeing a higher CPU usage. I certainly see at least 1 core pop up to about 50% CPU usage (roughly 8-10% total CPU on my 8-core processor) when I tell it to compress a large directory on my system...
I really think that hard drive seek times are your problem here, otherwise you need to specifically look at which cores are being worked while compressing, it may be that the compression routine is only able to be used by one thread at a time, though I would doubt this to be the case.