Linux – Why are cgroups mapped differently on these two systems by systemd

cgroupslinux-kernelsystemd

I have two systems, both running customized variants of OpenSuSE 12.2 (Mantis), both running exactly the same kernel. I get two very different outputs of /proc/self/cgroup or /proc/$$/cgroup on the two systems:

System A (or a stock OpenSuSE 12.1):

cat /proc/self/cgroup 
9:perf_event:/
8:blkio:/
7:net_cls:/
6:freezer:/
5:devices:/
4:memory:/
3:cpuacct,cpu:/
2:cpuset:/
1:name=systemd:/user/root/6

System B:

root@msx:/sys/fs/cgroup> cat /proc/self/cgroup 
9:perf_event:/
8:blkio:/
7:net_cls:/
6:freezer:/
5:devices:/
4:memory:/
3:cpuacct,cpu:/system/serial-getty@.service/ttyS0
2:cpuset:/
1:name=systemd:/system/serial-getty@.service/ttyS0

Why are lines 1 different and why is line 3 present in one and absent in the other? I can find no configuration differences between the two systems. They run the same version of systemd (systemd-44-10.1.1.x86_64). If I duplicate the sysctl values it has no effect. The boot options are the same. I've compared everything in /etc, /usr, and /lib to see if there are any relevant configuration differences. (There are different installed RPMs but none that lay down any system configuration files, and I think I removed all the custom RPMs.)

This is more than a curiosity factor because on System B, we cannot create a SCHED_RR thread, but on System A, we can. If I set DefaultController to NULL in /etc/systemd/system.conf, it works, and line 3 disappears (lines 1 remain different). It also works if I write the process ID of the invoking shell into /sys/fs/cgroup/cpu,cpuacct:

root@msx:/root> ./a.out
Creation of real-time thread FAILED - Operation not permitted
root@msx:/root> cat /proc/$$/cgroup
9:perf_event:/
8:blkio:/
7:net_cls:/
6:freezer:/
5:devices:/
4:memory:/
3:cpuacct,cpu:/system/sshd.service
2:cpuset:/
1:name=systemd:/system/sshd.service
root@msx:/root> echo $$ > /sys/fs/cgroup/cpu,cpuacct/tasks
root@msx:/root> ./a.out
root@msx:/root> cat /proc/$$/cgroup
9:perf_event:/
8:blkio:/
7:net_cls:/
6:freezer:/
5:devices:/
4:memory:/
3:cpuacct,cpu:/
2:cpuset:/
1:name=systemd:/system/sshd.service

Not sure why this works. My colleagues aren't satisfied that the problem is understood, as it shouldn't require this configuration change.

Kernel is 3.4.47, CONFIG_RT_GROUP_SCHED is enabled, CONFIG_AUTOGROUP is enabled (disabling still doesn't work but it does fail differently)

This is a spinoff of https://stackoverflow.com/questions/20412336/how-to-check-for-permissions-for-sched-setscheduler.

Best Answer

There is a system.conf configuration option, DefaultControllers, that controls which cgroup hierarchies are attached to. By default it's cpu. I set it to null and /proc/$$$/cgroup no longer lists the getty process under cpuacct,cpu, and the test program works. Why the same configuration file -- I was using the default which is in use on both systems -- produced two different results, I don't know. I'm not sure this is the best way to solve the problem or why it works. So I changed

#DefaultControllers=cpu

to

DefaultControllers=

in /etc/systemd/system.conf and it works.

Related Question