Windows 10 NTFS compression of copied files

compressionntfswindows

I'm experiencing an odd issue in Windows 10 that involves copying files that are compressed using the NTFS filesystem to another folder that has the compression flag set (so that when the file is copied, it is compressed in the destination folder).

The issue is that when the file is copied to the folder, the filesystem is not recognizing that the file is compressed, and appears to store on disk a copy of the original file and another copy of the file that is supposed to be what is actually stored (the compressed variant). Thus, the disk is recognizing that double the space is needed to store the file (e.g. if the file was 500MB, the remaining capacity for the disk will be 1000MB less).

Also, I've experienced this on several Windows 10 installs, so I'm wondering if anybody else has seen this.

Of course, restarting the computer seems to correct the reported free disk space remaining, but while in a session, I've had to decompress a file manually then recompress it to have the filesystem report the correct size. The system believes that much more disk space is being consumed that what is really there, and it is leading to insufficient disk space errors (in testing out the problem, I made a dummy partition and copied the compressed file to it).

Thanks for any help.

Best Answer

That is because for any NTFS compressed file being copied (or even accessed for that matter), first it has to be decompressed to make it readable for Windows. This is because the NTFS compression is done at the file system level, and at Windows OS level there is no difference in the way it reads the contents of the file. So when the compressed file is copied to the compressed destination folder, it is first decompressed at NTFS level, then copied to the destination folder at the OS level, and then it is again compressed back at the NTFS level to match the compression attribute of the destination folder (and it is not compressed back because the source file being copied was compressed to start with, however, in the case of moving a file it retains the compression attribute of the source file irrespective of the destination folder attribute). Now this temporary uncompressed version of the compressed file that was decompressed, copied, and then re-compressed, still exists in the destination drive until the computer is restarted or disk cleanup or ccleaner utility is used to cleanup system files. This is exactly why the double file size count was solved on restart. Therefore, for a compressed drive it is always recommended to set aside at least enough free space to fit an additional uncompressed version of the largest compressed file contained within the drive.

Related Solutions

Linux – Compressed disk image on Linux

Use a forensic analysis software like GUYMAGER (open source, sourceforge.net). It has a nice UI which allows to quickly create a compressed disk image of an entire hard disk.

Use "Advanced forensic image (.aff)" This creates a single, compressed file (well, it also creates an .info file).

To modify the default compression rate 1 (fastest, but least compression). If you have a fast computer with lots of cores, you can change this by creating /etc/guymager/local.cfg:

AffCompression = 3

9 is the best but slowest compression. 3 gives a good compression with good performance.

Update

Mounting isn't as simple as it seems. First of all, you need AFFLIB (Ubuntu: aptitude install afflib-tools). Now you can get the raw disk image with [affuse][3] <image> <mount-point>

But for some reason, mounting the raw image fails. parted says the first partition starts with 1048576B but

mount -t ext4 -o loop,ro,offset=1048576 /mnt/backup.raw /mnt/backup

fails with the usual useless mount error:

mount: wrong fs type, bad option, bad superblock on /dev/loop0,
   missing codepage or helper program, or other error
   In some cases useful info is found in syslog - try
   dmesg | tail  or so

and dmesg says:

EXT4-fs (loop0): VFS: Can't find ext4 filesystem

Windows – Should I compress hard drives containing large video files

No, you should not compress your hard drive.

Compressing your hard drive may save you some space. However, compression is a two-part process: compression and decompression. Although decompression isnt as taxing on the computer as compression, it still requires CPU, RAM, and disk cycles. This can cause video and audio playback to become stuttered. Disk compression is best used on files that you are not likely to frequently access.

What you should look at are video codecs and audio codecs. A codec is software that enables compression or decompression of digital media. Codecs are a can be both lossy and non-lossy compression formats, but that doesnt necessarily mean the video/audio will be noticeably degraded. Video and audio codecs can strip out data that the humans cant see or hear. The most common example are MP4 videos and MP3 audio. DVD and BluRay videos are compressed with codecs as is the music you buy from online stores.

Im not saying disk compression isnt a solution, its just not the right tool for the job. You dont use a steam roller to hammer a nail.

Best Answer

Related Solutions

Linux – Compressed disk image on Linux

Windows – Should I compress hard drives containing large video files

Related Question