Linux – How to use an SSD for caching, so the hard disks can spin down

bcachecachehard-disklinuxssd

I've got a file server with a bunch of disks running together in a BTRFS pool and want to add an SSD for caching. I'm not mainly after speeding up things, but want to catch the regular small accesses to be able to shut down the hard disks most of the time of not under heavy use (not running them 24/7 will save energy and should make the disks last longer).

As far as I know, there are currently two SSD caching techniques implemented in Linux, dm-cache and bcache. dm-cache is still told to be more efficient, but development is going on for both and I don't need to tune for the absolute maximum efficiency.

Reading bcache's documentation, I came upon these options:

writeback_delay: When dirty data is written to the cache and it
previously did not contain any, waits some number of seconds before
initiating writeback. Defaults to
30.

writeback_percent: If nonzero, bcache tries to keep around this
percentage of the cache dirty by throttling background writeback and
using a PD controller to smoothly adjust the rate.

writeback_running: If off, writeback of dirty data will not take
place at all. Dirty data will still be added to the cache until it
is mostly full; only meant for benchmarking. Defaults to on.

Setting a large enough value for writeback_delay seems to do the job for me: Only write back once an hour, or (I assume that this would happen) if the cache is running full.

Is this a reasonable setup, and do I have consider anything else to succeed at spinning down the disks? I'm also fine with going a completely different route if it fulfills my requirements.

It seems @gorkypl is looking for another solution on a similar problem, but is having different requirements and environment and didn't receive an answer yet, either.

Best Answer

I think your approach is too complicated.

read caching: Nothing to do here. If you have enough ram this is done automatically in linux.

write caching Basically this is what you want. But if the writes go to disk in the end that will cause a wakeup, too.

So you could put affected filesystems directly onto ram-disk /dev/shm or ssd instead.

power saving: I do not think that frequent spin down/up will save power. On the contrary the disks might die earlier so there is additional energy consumption for the production process. Also spin up is very power intensive.

Related Solutions

Linux – Cannot attach cache device to backing device

I managed to fix this by re-creating the superblock on /dev/sdc4. Looks like the --block 4k --bucket 2M was incorrect and that's why the cache device wasn't attaching.

I cleared the superblock, then ran:

make-bcache -C /dev/sdc4

Now when I did:

echo 'uuid' > /sys/block/bcache0/bcache/attach

it worked!

Linux – Disable writeback cache throttling – tuning vm.dirty_ratio

The problem is not the dirty-throttling, but the immediate syncing. You can check it with the bo column in vmstat 1 - I get:

   -----io----
   bi    bo
    0     0
    0     0
    0  4000
    0     0
    ...

after dd if=/dev/zero of=16/test bs=4k count=1000. (The vmstat output is just 1K-"units" - /proc/diskstats has 512B-units)

Adding conv=notrunc avoids that, as does a rm before. See sourcejedi's comment.

The ext4 mount option noauto_da_alloc exactly is meant for that: no extra sync for these "write-file-anew" operations. man ext4 has some good examples. The question goes back to what should "write" or "save" file mean in an editor, and how to make that (power-) fail-safe.

You can also check the other side: the dirty pages growing:

]# grep nr_dirty /proc/vmstat      
nr_dirty 7886
nr_dirty_threshold 427666
nr_dirty_background_threshold 170815

The thresholds are the ratios applied to the current available memory. Only since I quite disabled the periodic writebacks I see more than a dozen dirty pages.

vim

It is really a complex situation; vim has at least two crucial options: fs and wb: filesync after write and overwrite via backup.

On ext2 it is enough to set nofs in vim. On ext4 (with default delayed allocation) the noauto_da_alloc mount option is also needed to prevent an immediate syncing, this time by the filesystem.

(with dd you have the option to (over)write only a part of a file, unlike cp or vim)