I have a 40 v core machine in which a java process running. I can see that this process is running in cpu6 all the time.
$watch ps -o pid,psr,comm -p 4076
Every 2.0s: ps -o pid,psr,comm -p 4076 Mon Apr 11 02:24:12 2016
PID PSR COMMAND
4076 6 java
$ sudo taskset -c -p 4076
pid 4076's current affinity list: 0-39
$ sudo taskset -p 4076
pid 4076's current affinity mask: ffffffffff
This is how top command for cpu utilisation looks
$ top -c
top - 02:27:37 up 47 days, 10:49, 9 users, load average: 2.83, 2.89, 3.01
Tasks: 470 total, 1 running, 469 sleeping, 0 stopped, 0 zombie
%Cpu0 : 36.0 us, 8.3 sy, 0.0 ni, 40.7 id, 8.7 wa, 0.0 hi, 6.3 si, 0.0 st
%Cpu1 : 1.3 us, 0.7 sy, 0.0 ni, 97.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu2 : 0.3 us, 0.3 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu3 : 0.7 us, 0.7 sy, 0.0 ni, 98.0 id, 0.7 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu4 : 1.4 us, 1.0 sy, 0.0 ni, 97.3 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu5 : 1.0 us, 1.3 sy, 0.0 ni, 97.0 id, 0.7 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu6 : 0.3 us, 0.3 sy, 0.0 ni, 99.0 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu7 : 0.3 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu8 : 0.3 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu9 : 1.0 us, 1.0 sy, 0.0 ni, 97.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu10 : 0.7 us, 0.7 sy, 0.0 ni, 98.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu11 : 0.7 us, 0.0 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu12 : 0.7 us, 1.3 sy, 0.0 ni, 97.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu13 : 0.3 us, 0.7 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu14 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu15 : 0.7 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu16 : 0.3 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu17 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu18 : 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu19 : 1.7 us, 0.3 sy, 0.0 ni, 97.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu20 : 0.7 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu21 : 0.7 us, 0.0 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu22 : 0.3 us, 0.0 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu23 : 0.7 us, 0.3 sy, 0.0 ni, 98.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu24 : 1.3 us, 0.7 sy, 0.0 ni, 98.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu25 : 2.0 us, 0.3 sy, 0.0 ni, 97.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu26 : 0.7 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu27 : 0.3 us, 0.7 sy, 0.0 ni, 98.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu28 : 0.3 us, 0.7 sy, 0.0 ni, 98.7 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu29 : 0.7 us, 0.7 sy, 0.0 ni, 98.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu30 : 1.0 us, 0.3 sy, 0.0 ni, 98.3 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu31 : 1.0 us, 0.7 sy, 0.0 ni, 98.0 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu32 : 0.7 us, 0.0 sy, 0.0 ni, 99.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu33 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu34 : 0.7 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu35 : 0.7 us, 0.0 sy, 0.0 ni, 99.0 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st
%Cpu36 : 0.0 us, 0.3 sy, 0.0 ni, 99.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
Cpu6 is idle most of the time as you see it in top command. So scheduling process on same processor is possible and it will help to reduce cache invalidation. But how is scheduler doing all this? Is it keeping a track of each process and their last executed processor?
Best Answer
Yes, Linux's scheduler keeps track of where each thread was last scheduled, and favors keeping a thread on the same CPU if it can. You guessed the reason: migrating a thread from a CPU to another is more expensive than keeping it on the same CPU. There's even more to it: the scheduler knows about multiple levels of cache coherency, and tries not to migrate threads too far: moving a thread inside the same hyperthreaded core is very cheap, moving to another core on the same CPU chip is less cheap, moving to another CPU chip is more expensive, etc. The tool for this is scheduling domains.