Can the “du” program be made less aggressive

disk-usageiolimit

We have a regular job that does du summaries of a number of subdirectories, picking out worst offenders, and use the output to find if there are things that are rapidly rising to spot potential problems. We use diff against snapshots to compare them.

There is a top level directory, with a number (few hundred) of subdirectories, each of which may contain 10's of thousands of files each (or more).

A "du -s" in this context can be very IO aggressive, causing our server to bail its cache and then massive IO spikes which are a very unwelcome side affect.

What strategy can be used to get the same data, without the unwanted side effects?

Best Answer

Take a look at ionice. From man ionice:

This program sets or gets the io scheduling class and priority for a program. If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process.

To run du with the "idle" I/O class, which is the lowest priority available, you can do something like this:

ionice -c 3 du -s

This should stop du from interfering with other process' I/O. You might also want to consider renicing the program to lower its CPU priority, like so:

renice -n 19 "$duPid"

You can also do both at initialisation time:

nice -n 19 ionice -c 3 du
Related Question