FreeBSD zfs timeout errors

freebsdzfs

I have FreeBSD 8.2 with ZFS v28. 8 cpu cores, 16Gb RAM.
Two disk drives (1.5Tb WD Green) on ZFS mirror raid.

My system strangely uses memory.
With a lot of free (wired) memory, it starts to use swap and then doesn't clean it! Swap usage keeps growing. munin-graph swap

And then I see on my kvm these messages:
kvm log

All file reads or writes seem very slow. When I use ssh and, for example, do nano /etc/rc.conf it pauses for 3-5 seconds before it works.

Will my disk drives die soon, or is the problem elsewhere?

Best Answer

You have read the ZFS Tuning Guide, right?

Early versions of ZFS had more than a few issues with memory management, especially when using the dedupe and/or compression options. For the longest time, a simple recipe for ZFS hangs was to set compression=gzip and start copying data. Later versions of ZFS are much better. If you can, you should upgrade to FreeBSD 8.4 or 9.1.

I have a very comparable system at home (16GB RAM, 2 x 3TB SATA disks in ZFS mirror) that's been rock solid since upgrading to FreeBSD 9.1. I have compression enabled and these settings in /boot/loader.conf:

# attempts to get ZFS more stable
vfs.zfs.arc_max="4096M"
kern.maxvnodes=250000
vfs.zfs.prefetch_disable=1
vm.kmem_size="1024M"
vm.kmem_size_max="1024M"

Another thing to check is your motherboard settings for the SATA ports. Initially, my motherboard had the SATA ports set to IDE mode and performance under ZFS was awful, making the memory exhaustion issues worse. After switching them to AHCI mode, the disks perform quite well.

Related Question