Ok, I found it. In the ProcessExplorer you can right click on the process, select properties. From there, select Threads Tab, it shows all the threads running inside the process with "dynamic priorities".
That's what I was looking for.
Ready-Made Tool
A third-party tool to set process I/O priorities, either by manual user action or automation and persistence, would be Process Lasso. It can set almost any process setting, and do so persistently, or by rules (e.g. when this happens, change I/O priority of that).
Programmatic Answer
The answer is a simple API call to an NT Native API, NTSetInformationProcess. It is defined as the class 'IoPriority' in the 2008 DDK. In NT6 only two levels supported: Very Low (background) and Normal. There are technically 5 levels defined -- Critical, High, Normal, Low, and Very Low. The Critical level is reserved for paging operations. No data on the others.
However, the proper way to use the I/O priority is only to start a 'background mode' via a call to the Windows API SetProcessPriorityClass, supplying PROCESSS_MODE_BACKGROUND_BEGIN. This will cause it to use the very Low (background) I/O priority, along with an IDLE CPU priority class. When done, revert via PROCESS_MODE_BACKGROUND_END.
The SetThreadPriority API will also allow this same background mode for specific threads. Here you'd supply THREAD_MODE_BACKGROUND_BEGIN/END.
Best Answer
Yes. I tested it. IO priority is inherited just like CPU niceness. This is probably what you want. If it isn't, you can explicitly specify the IO priority of child processes with the ionice command.