Similar to the question I asked a while back. This one's a little easier? I can see there's volume management logic in BTRFS for removing/adding/rebalancing physical drives, but is there a tool to manually move extents/blocks off a particular physical device and onto another? Seems like it would be important for files that should be created quickly (like backup files) but can be moved to slower storage on the backend.
Equivalent of pvmove for BTRFS
btrfsfilesystems
Related Solutions
From wiki:
Extent based file storage 2^64 byte == 16 EiB maximum file size Space-efficient packing of small files Space-efficient indexed directories Dynamic inode allocation Writable snapshots, read-only snapshots Subvolumes (separate internal filesystem roots) Checksums on data and metadata Compression (gzip and LZO) Integrated multiple device support RAID-0, RAID-1 and RAID-10 implementations Efficient incremental backup Background scrub process for finding and fixing errors on files with redundant copies Online filesystem defragmentation
Explanation for desktop users:
- Space-efficient packing of small files: Important for desktops with tens of thousands of files (maildirs, repos with code, etc).
- Dynamic inode allocation: Avoid the limits of Ext2/3/4 in numbers of inodes. Btrfs inode limits is in a whole different league (whereas ext4's inodes are allocated at filesystem creation time and cannot be resized after creation, typically at 1-2 million, with a hard limit of 4 billion, btrfs's inodes are dynamically allocated as needed, and the hard limit is 2^64, around 18.4 quintillion, which is around 4.6 billion times the hard limit of ext4).
- Read-only snapshots: fast backups.
- Checksums on data and metadata: essential for data integrity. Ext4 only has metadata integrity.
- Compression: LZO compression is very fast.
- Background scrub process to find and to fix errors on files with redundant copies: data integrity.
- Online filesystem defragmentation: autodefrag in 3.0 will defrag some types of files like databases (e.g. firefox profiles or akonadi storage).
I recommend you the kernel 3.0. Also btrfs is a good FS for SSD.
The replace command doesn't make a backup of sda1, it replaces sda1 with sdb1 in the filesystem, but since it's a one device filesystem and btrfs doesn't bother wiping the data from sda1 when it replaces it they end up being indentical copies of the filesystem. However you do NOT want to do this as both will have the same UUID, and currently it's not safe to mount two btrfs filesystems with the same UUID as it can cause MASSIVE DATA CORRUPTION (see btrfswiki's Gotchas page). If you want to use btrfs's incremental backup feature you should format you're backup drive /dev/sdb1 to a new btrfs filesystem. Then you should make a read-only snapshot of watever subvolume(s) you want to backup on your filesystem by using
btrfs su sn -r @subvolume-name @subvolume-name-RO
on each subvolume. Then you should mount the blank btrfs filesystem and run
btrfs send /path/to/@subvolume-name-RO | btrfs rec /path/to/backup-directory/
This will be the first send and btrfs will have to transfer all of the data this time. Next time you want to send a backup to this drive you can use incremental sends to only send what data has changed since the previous backup you sent. It will also use Copy On Write so you'll save a lot of space as well. Just make sure you keep the latest snapshot on both filesystems. When it's done you can rename the sent snapshot to whatever you want.
Now if you want to send another snapshot just rename the orignal one and take a new snapshot with something like
mv @subvolume-name-RO @subvolume-name-RO-old
btrfs su sn -r @subvolume-name @subvolume-name-RO
Then you can use send to send the latest snapshot using
btrfs send -p @subvolume-name-RO-old @subvolume-name-RO | btrfs rec /path/to/backup-directory/
and if the previous snapshot still exists on your backup drive it will send your new snapshot by only having to copy whatever changes you've made since the previous one.
Best Answer
The redistribution of files when removing a device is transparent. I looked at the source ( a 1 Gb git clone) but I have never been able to find anything in the user-land utilities (now bundled in
btrfsIIRC) to do this without actually removing (btrfs device delete) and re-adding (btrfs device add) the device. I specifically looked at mapping subvolumes to specific devices without results.The removing of a device takes time equivalent to the amount of data to be moved, during which the device is not used for new data. Also, there is no control to leave frequently accessed items on the device. So that was not a useful option.
I have my setup now so that the slower 'backend' is a separate filesystem to which specific data is move based on file type and access information. That is, of course, not as convenient as having one big filesystem with respect to running out of space (at least 'backend' storage is relatively cheap).
If you have not already done so you might also want to look at this entry on the btrfs Wiki.