Ubuntu – All services of a user are killed when running multiple services under this user with systemd

systemdUbuntu

We're using Ubuntu 16.04 LTS and want to use multiple Tomcat installations which should start at boot time. One of these Tomcats would host a Jenkins which will deploy a webapp onto the other tomcat and restart it.

To start the services, we added systemd service scripts.
What we noticed is that when one tomcat is stopped or killed, the other is stopped as well.

We reduced this to two simple scripts that only use /usr/bin/yes:

Unit A

[Unit]
Description=A
After=syslog.target network.target

[Service]
Type=simple

ExecStart=/usr/bin/yes
ExecStop=/bin/kill -15 $MAINPID

User=tomcat8
Group=tomcat8

[Install]
WantedBy=multi-user.target

Unit B

[Unit]
Description=B
After=syslog.target network.target

[Service]
Type=simple

ExecStart=/usr/bin/yes
ExecStop=/bin/kill -15 $MAINPID

User=tomcat8
Group=tomcat8

[Install]
WantedBy=multi-user.target

What happens:
When a service is killed (kill -9), both services are gone afterwards.

  1. Why are both services killed? How can we prevent this?
  2. Is running more than one service under a single user discouraged, or is this good practice?

EDIT: For clarification – we did also try to do the same when launching the tomcats without systemd. In this instance, the behaviour was as expected: only the killed service was stopped while the other lived on.

EDIT2: The user is not a front-end user that logs in/out at all. It's purely a system user to restrict access of the services.

Best Answer

The changelog for systemd (v230) says:

systemd-logind will now by default terminate user processes that are part of the user session scope unit (session-XX.scope) when the user logs out. This behavior is controlled by the KillUserProcesses= setting in logind.conf, and the previous default of "no" is now changed to "yes". This means that user sessions will be properly cleaned up after, but additional steps are necessary to allow intentionally long-running processes to survive logout.

So this is default behaviour. It also explains what to do to undo the change: logind.conf, set KillUserProcesses= to no (and --without-kill-user-processes option to configure)

But the changelog also includes a ...

While the user is logged in at least once, user@.service is running, and any service that should survive the end of any individual login session can be started at a user service or scope using systemd-run. systemd-run(1) man page has been extended with an example which shows how to run screen in a scope unit underneath user@.service. The same command works for tmux.

and

After the user logs out of all sessions, user@.service will be terminated too, by default, unless the user has lingering enabled. To effectively allow users to run long-term tasks even if they are logged out, lingering must be enabled for them. See loginctl(1) for details. The default polkit policy was modified to allow users to set lingering for themselves without authentication.

That one is more important since it uses the default (kill'm all) with a way to provide exceptions: enable lingering.

Some more info:

Related Question