Relationship between IO scheduler and cpu/process scheduler

filesystemsiokernelscheduling

The concepts of IO scheduler and CPU scheduler confuse me. Below are my understanding:

  1. Linux uses CFS scheduler + nice values by default to schedule processes.
  2. Each process has an IO queue.
  3. There is an IO scheduler kernel thread.
  4. IO scheduler is in the block level, not in the file level.
  5. IO scheduler is a module of the file system.

Questions:

What is relationship between IO scheduler and CPU scheduler? Conceptually, it seems to me that CPU scheduler is superior over IO scheduler. CPU scheduling happens first. IO scheduler is a thread itself and subject to CPU scheduling.

A contrived scenario looks like this:

Step 1: CPU scheduler picks a process P1 to execute

Step 2: P1 puts IO requests in its own IO queue

Step 3+: CPU scheduler picks other threads to run. (Assuming no process has IO other than P1)

….(after a while)

Step n: CPU scheduler picks the IO scheduler thread to run.

Step n+1: IO scheduler thread 'notices' P1 has IO requests queued up and issues those requests to disk.

Does my understanding and the scenario make sense?

Best Answer

Let's start with the IO scheduler first. There's a IO scheduler per block device. Its job is to schedule (order) the requests that pile up in the device queue. There are three different algorithms currently shipped in the linux kernel: deadline, noop and cfq. cfq is the default, and according to its doc:

The CFQ I/O scheduler tries to distribute bandwidth equally among all processes in the system. It should provide a fair and low latency working environment, suitable for both desktop and server systems

You can configure which scheduler governs which device via the scheduler file corresponding to your block device under /sys/ (You can issue the following command to find it: find /sys | grep queue/scheduler).

What that short description doesn't say is that cfq is the only scheduler that looks at the ioprio of a process. ioprio is a setting that you can assign to the process, and the algorithm will take that into account when choosing a request before another. ioprio can be set via the ionice utility.

Then, there's the task scheduler. Its job is to allocate the CPUs amongst the processes that are ready to run. It takes into account things like the priority, the class and the niceness of a give process, as well as how long that process has run and other heuristics.

Now, to your questions:

What is relationship between IO scheduler and CPU scheduler?

Not much, besides the name. They schedule different shared resources. The first one orders the requests going to the disks, and the second one schedules the 'requests' (you can view a process as requesting CPU time to be able to run) to the CPU.

CPU scheduling happens first. IO scheduler is a thread itself and subject to CPU scheduling.

It doesn't happen like the the IO scheduler algorithm is run by whichever process is queuing a request. A good way to see this is to look at crashes that have elv_add_request() in their path. For example:

 [...]
 [<c027fac4>] error_code+0x74/0x7c
 [<c019ed65>] elv_next_request+0x6b/0x116
 [<e08335db>] scsi_request_fn+0x5e/0x26d [scsi_mod]
 [<c019ee6a>] elv_insert+0x5a/0x134
 [<c019efc1>] __elv_add_request+0x7d/0x82
 [<c019f0ab>] elv_add_request+0x16/0x1d
 [<e0e8d2ed>] pkt_generic_packet+0x107/0x133 [pktcdvd]
 [<e0e8d772>] pkt_get_disc_info+0x42/0x7b [pktcdvd]
 [<e0e8eae3>] pkt_open+0xbf/0xc56 [pktcdvd]
 [<c0168078>] do_open+0x7e/0x246
 [<c01683df>] blkdev_open+0x28/0x51
 [<c014a057>] __dentry_open+0xb5/0x160
 [<c014a183>] nameidata_to_filp+0x27/0x37
 [<c014a1c6>] do_filp_open+0x33/0x3b
 [<c014a211>] do_sys_open+0x43/0xc7
 [<c014a2cd>] sys_open+0x1c/0x1e
 [<c0102b82>] sysenter_past_esp+0x5f/0x85

Notice how the process enters the kernel calling open(), and this ends up involving the elevator (elv) algorithm.

Related Question