Ubuntu – How to control the time that users spend on

The usual reason for hitting this issue, is that the process is blocking in page faults. These are reads or possibly writes to files performed through a memory mapping aka mmap(). You may have noticed some mmap() in the trace of system calls.

If you had used the /usr/bin/time program instead of the time shell builtin, you might also have noticed:

0.04user 0.10system 0:02.29elapsed 6%CPU (0avgtext+0avgdata 40464maxresident)k
73632inputs+0outputs (376major+1081minor)pagefaults 0swaps

major pagefaults are the ones that require filesystem IO. minor pagefaults are much less significant (probably only a "TLB miss").

I suspect inputs are the total number of pages read. Currently, I think file mapped pages are always the same size. 4096 bytes in most cases, but you can check getconf PAGESIZE.

So this represents ~290 megabytes, read at something over 100 megabytes per second, a standard speed for a hard disk like mine. Mystery solved!

Note also, you are assuming that you have a whole free CPU for this process. Otherwise, the process could simply be blocked waiting for other processes to yield the CPU.

strace only shows when the process enters (and then leaves) the kernel due to a system call. Or when a unix signal is delivered. However there are other types of interrupts which strace does not show at all. So these include

• Page faults.
• The timer interrupt. This is used to switch to a different process, when the current one has exhausted its allocated time slice on the CPU.

Given the context of wanting to reboot a Linux system, I would take a multi-pronged approach.

First, disable future logins by creating an /etc/nologin file. You could leave it blank or enter informative text in there, such as:

"Logins to this system have been temporarily disabled in preparation for a server reboot, scheduled for (time and date). Please try again after (expected end time)."

Don't forget to remove /etc/nologin when you're done!

Additionally, since a reboot will clear all processes, whether they're interactive or not, I would use ps to look for processes owned by users. This will take some manual investigation to determine whether the processes are worth keeping or not, but should narrow the field some. I've hard-coded 1000 here as the value of UID_MIN from /etc/login.defs as the cutoff for "system" vs "user" UIDs. If any of your users have UIDs below 1000, you'll need to adjust that number.

ps -eo pid,uid,args | awk '$2 >= 1000'

Of course, you could adjust the ps columns to taste, perhaps to add the translated username and process start time (ps -eo pid,uid,user,start,args) or others -- just be careful to keep the ps UID and awk field in sync with each other.

To get the list of unique user names, use:

ps -eo user,uid | awk 'NR>1 && $2 >= 1000 && ++seen[$2]==1{print $1}'