Debian – How to prevent disk I/O from taking over the CPU

debianfreezeiolinux

I run Debian Jessie on a quad-core i7 processor, which results in 8 cores being shown thanks to hyperthreading. On disk-heavy loads, all eight cores are maxed out with io-wait, as shown by htop. This results in a very sluggish interface (I use Gnome), and the system sometimes freezes completely until the I/O is complete.

How should I go about diagnosing this? Which settings can I tune to prevent this from occurring?

I use a single HDD, and a S.M.A.R.T check shows that its healthy.

Edit: To clarify a few things, I'm not interested in speeding up the I/O. I'd like to just have it run in the background while I do something else. Upgrading to an SSD is currently not an option for me.

Best Answer

There are several ways to tune I/O performance.

  • file system choice - choose a file system depending on what your typical data looks like - some file systems handle better lots of smaller files, some do well for large files, some don't scale well when accessing (especially creating) multiple files at once. Defragmenting might help on spinning plate drives (not so much on flash based devices, but the difference should still be there).

  • file system tweaking - modern file systems have loads of options (both create- and mount-time) which push performance of the file system in one direction or another. Prime example can be how journalling is done.

  • DMA setting - while this has been done automagically since ages, it never hurts to check it (hdparm -i is your friend here).

Linux specifically (but might be similar for other kernels as well):

  • THP - Transparent Huge Pages used to cause performance degradation. IIRC the problem has been fixed (at least to some extent) some time ago. If copying to/from USB is the issue, you probably should look in this direction. disable transparent hugepages might serve as a good starting point as for how to disable THP.

  • kernel I/O scheduler can influence the performance a big deal. Some people advocate the use of the Deadline scheduler over CFQ which is more complex and sometimes allegedly tends to overthink things.

Apart from that, in some scenarios heavy I/O just happens. When you start copying files from USB memory card reader to hard drive, from hard drive to a USB flash disk, browse the web (some browsers might be fsync() heavy to make sure things are recoverable in case of a crash), download emails, run a P2P client and play a movie all at the same time, the physical disk and the file system are just going to become the bottleneck. In some cases a carefully constructed (read with appropriate stripe size and number of constituent drives) RAID0 might help to alleviate the problem.

Related Question