Debian – Is /etc/rc.local a good place to run a script before any (normal) user can log in

I'm not sure what that means. Does that mean it executes multiple times? Isn't that a bad thing?

No, this script execute only one time, at the end of each run level from 2 to 5. In the Debian RunLevel system, multiuser runlevel is defined from runlevel 2 through runlevel 5. A default Debian installation does not make any differences between them.

In Debian, default runlevel is 2. You can check/change default runlevel by reading/editting /etc/inittab:

$ grep initdefault /etc/inittab 
id:2:initdefault:

Anyway, I'm looking for advice (or possibly scripts) for a way to start the server automatically on boot

With the instruction from documentation. I think using rc.local trick is enough to start automatically on boot. If you want more complicated controls, you should write your own init script for smuxi-server.

You can read an example here or getting a script from /etc/init.d/ directory for reference:

#!/bin/bash

USER=michael
GROUP=michael
PIDFILE=/var/run/smuxi.pid

case "${1:-''}" in
  'start')
        start-stop-daemon -S -c $USER -g $GROUP --make-pidfile --pidfile $PIDFILE --background --startas /usr/bin/smuxi-server -v
        ;;
  'stop')
        start-stop-daemon -K --pidfile $PIDFILE -v
        ;;
  *)
        echo "Usage: $SELF start|stop"
        exit 1
        ;;
esac

This script does not use LSB Specification, but it's usable. You can read man start-stop-daemon for more understanding.

Note

• Purpose and typical usage of /etc/rc.local
• Init Script Functions

Yes, there was something obvious that I was missing.

Systemd is all about concurrent start of services, so the cloud-init script is run at the same time the apt-daily.service is triggered. By the time cloud-init gets to execute the user-specified payload, apt-get update is already running. So the attempts 2. and 3. failed not because of some namespace magic, but because they altered the system too late for apt.systemd.daily to pick the changes up.

This also means that there is basically no way of preventing apt.systemd.daily from running -- one can only kill it after it's started.

This "user data" script takes this route::

#!/bin/bash

systemctl stop apt-daily.service
systemctl kill --kill-who=all apt-daily.service

# wait until `apt-get updated` has been killed
while ! (systemctl list-units --all apt-daily.service | egrep -q '(dead|failed)')
do
  sleep 1;
done

# now proceed with own APT tasks
apt install -y python

There is still a time window during which SSH logins are possible yet apt-get will not run, but I cannot imagine another solution that can works on the stock Ubuntu 16.04 cloud image.