Linux – Have systemd not kill your service if it is in a state it should not be killed

linuxsystemctlsystemd

I have a question regarding the configuration of a systemd service.

The service application is an application the controls a machine. Within the application the SIGINT, SIGTERM, SIGQUIT and SIGHUP are captured. When the machine is "RUNNING", these signals are ignored and the application is not exited. If the machine is in "STOPPED" mode, the controlling application is exited.

We want to boot this application together with Linux, so we added the application as a systemd service.

We have the following configuration so far:

[Unit]
Description=Machine control service
After=network.target

[Service]
Type=simple
User=simplemachine
Group=simplemachine
CPUSchedulingPolicy=other
LimitRTPRIO=80
LimitRTTIME=infinity

WorkingDirectory=/opt/simplemachine/bin/
ExecStart=/opt/simplemachine/bin/simplemachine
KillMode=none
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Now I have the following questions.
When I perform:

sudo systemctl stop machine.service

I would like systemctl to send a SIGTERM, this way the application is only stopped when it is allowed to stopped.
Also when the application does not stop. It would be nice that systemctl does not kill the process, but for example returns some kind of fault or timeout code meaning that it may not stop the process.

How can I achieve this with the new systemd system?

Best Answer

This answer is primarily based on the documentation for systemd.kill, but has been updated after doing some tests. It is admittedly not a perfect solution to this problem.

By setting SendSIGKILL=no in your unit file, it is possible to prevent the process from being killed. To allow the initial SIGTERM to be sent, you will likely need to restore the KillMode option to its default value, which is control-group.

With these settings, running systemctl stop machine.service should work like this:

  1. Because there are no ExecStop= commands specified in the unit file, a SIGTERM is sent to the process.

  2. After a period of 90 seconds (DefaultTimeoutStopSec), systemd considers terminating the process forcefully.

  3. Because SendSIGKILL is set to no, SIGKILL (FinalKillSignal) is not sent to the process, and the process continues running.

In effect, the only signal sent to the process on systemctl stop will be SIGTERM. Since the handling of SIGTERM is handled within the application itself, systemctl stop should work as intended: stops the application when the remote machine is down, times out when the remote machine is up.

The problem with this approach is noted by Michał Politowski in the comments. Namely, systemd will consider the unit to be failed once the stop timeout expires. This doesn't affect the process itself, but it will alter systemd's perspective of the process. If you issue another 'systemctl start' command while the unit is in this state, you'll end up with two processes.

The KillMode=none option that you already used avoids the process killing logic altogether. However, the results are similar to this approach. The state of the unit changes to inactive, while the processes continue to run.

As an additional note, the 90 seconds timeout can be configured with TimeoutStopSec.

Related Question