Get size of btrfs directory which may contain subvolumes

btrfs

The usual way to get directory size is:

du -hs /path/to/directory

However, it's not working and I suspect it's a btrfs issue. It's reporting that the size of a subdirectory with submodules is more than the total size of the disk itself.

However I'm able to check using gnome disks utility that the disk is more than 60% empty.

On further inspection by varying n in

du -h --max-depth=n /path/to/folder

I'm able to narrow large subdirectory sizes to the btrfs subvolumes which is why I think this issue is related to subvolumes

Best Answer

Yes, it's probably a BTRFS related issue.

As a general rule, classic du is not reliable with BTRFS, and quite often can give seemingly nonsensical results like this. This ultimately arises from the fact that stat() (the system call that du uses to see how much space each file is using on-disk) doesn't know or care about reflinks, snapshots, transparent compression, or pretty much anything else that BTRFS provides beyond the standard POSIX semantics.

The impact of this is that du will count any blocks shared between files via reflinks (which includes any blocks that have been deduplicated, blocks shared because of files being copied using the clone ioctl, and any that are part of snapshots) once fore each file that holds a reference to that block, and therefore du can show apparent disk usage far greater than actual space usage.

I would suggest using btrfs filesystem du instead of du, as it will properly count all that shared space usage.

Note also that GNOME Disk Utility and df are also not entirely reliable when dealing with BTRFS, but for different reasons (they properly account for reflinks, but don't understand the two-stage allocator BTRFS uses, so they can actually report the disk as mostly empty even if you can't write anything to it). As a result of this, they will quite often show different usage values from those you would get by running du on the root of the filesystem. (and it gets even worse if you use du -x, because that won't cross subvolume boundaries since they look like mount points).

Related Solutions

Running du command reports infinity as disk usage

I found this bug report where a user had the same issue on a btrfs system. It turned out that the problem came from a specific .xml file whose size was incorrectly reported as 16 exabytes.

In the case of the person who filed the bug, the issue was a filesystem corruption and the problem was solved by moving all files to /tmp (which was a tmpfs), deleting the originals and then copying them back again:

mkdir /tmp/foo &&
cp -rv ~/* /tmp/foo &&
rm -r ~/* &&
mv /tmp/foo/* ~/

Simple way to move btrfs subvolumes to a different btrfs filesystem

I am pretty sure, you won't get around btrfs-send/btrfs-receive. Another option would be, to copy the subvolumes and then deduplicate them on the destination, but this would take ages.

For the readonly problem: as these are daily snapshots, arent most of the volumes read-only anyway?

I would do a

$btrfs subvolume create /subvolumeA/source
$for i in /filesystemA/subvolume*; do btrfs subvolume snapshot -r $i /filesystemA/source/$i; done

Now you have all subvolumes under /filesystemA/source/ readonly.

Now you could send them in a loop incrementaly with btrfs-send with the parent statement to the destination.

If some of the snapshots should be writeable again, you can copy the content of these to a new writeable subvolume using cp --reflink.

Best Answer

Related Solutions

Running du command reports infinity as disk usage

Simple way to move btrfs subvolumes to a different btrfs filesystem

Related Question