Removing files on ZFS does not free disc-space

disk-spacesolariszfs

I'm trying to free space on a ZFS by removing files. They are large (>1GB) so it should show in the numbers. Unfortunately, after unlinking the files, df does not show any increase in available space. It seems as if the removed files have hardlinks somewhere (but ls shows hardlink counters of 1 for all files unless I create a hardlink manually (which I tested)).

The script removing the files opens, reads, and closes these files (I checked that), so there are no open file handles. Afterwards it uses unlink() to remove the files. ls also does not show them anymore. Just the available disk space does not increase.

We are using the ZFS feature of snapshots. Maybe this is a reason why nothing is freed?

Any ideas or hints to documentation concerning my problem are welcome.

Best Answer

According to Oracle's zfs documentation,

A snapshot is a read-only copy of a file system or volume. Snapshots can be created almost instantly, and they initially consume no additional disk space within the pool. However, as data within the active dataset changes, the snapshot consumes disk space by continuing to reference the old data, thus preventing the disk space from being freed.

If you want the disk space to be freed, you will have to clear out the snapshots referencing that file. You may also want to read up on copy-on-write, which is how zfs works.

I don't actually have any experience with zfs, but I make use of btrfs. btrfs does snapshots in a similar manner to zfs and the exact same 'issue' appears there. Of course, as I do not want a deleted file in the current working set to be removed from a snapshot, I would certainly expect that the available disk space would remain constant when removing a file. After all, the file isn't actually removed until all references (across all snapshots) are removed.

Eighty percent full (twenty percent free)

http://www.ustream.tv/recorded/25859777 audible around 33:00 on the timeline, in response to Eric Sproul's case:

… the Delphix product … to the user was eighty percent. So, I mean a lot of it depends on the workload but we would definitely … I think four percent would be extreme for any …

… And performance would suck.

– sounds like Matt Ahrens (moderating) at the 2012 Illumos ZFS Day.

Plus: recently rediscovered by me, from two years ago:

Oracle Solaris ZFS Performance: 10 Easy Tips (2010-04-28)

Under #8 Leave Enough Free Space:

… As a rule of thumb, don't let your pool become more full than about 80% of its capacity. Once it reaches that point, you should start adding more disks so ZFS has enough free blocks to choose from in sequential write order.

Shell Script – Handling Whitespaces in ZFS Dataset Names

Not splitting at spaces

bash's for loop splits the argument at all whitespace characters by default. You could somehow escape each line - or simply switch to another delimiter character. cut will return one value per line, so there's a newline in between which we can choose.

Watch out for the double quotes in zfs get all, so each $dataset doesn't get split up, too.

#!/bin/bash
IFS=$'\n'
for dataset in `zfs list -H | cut -f 1`
do
  zfs get all "$dataset"
done

Resetting IFS

Afterwards, you might want to reset IFS to the value before, store it to some temporary variable.

OLD_IFS=$IFS
# the whitespaces-sensitive lines
IFS=$OLD_IFS

Best Answer

Related Solutions

Macos – What free space thresholds/limits are advisable for 640 GB and 2 TB hard disk drives with ZEVO ZFS on OS X

Eighty percent full (twenty percent free)

Shell Script – Handling Whitespaces in ZFS Dataset Names

Not splitting at spaces

Resetting IFS

Related Question