Bash – script can’t write to systemd journal

First, the ability to use strings stems from a mere coincidence in that journald does not compress small (below 64KiB in current implementation) fields. This is not a supported way to read journals, and in the archwiki it really should be marked as such. It is more of a last-ditch recovery method.

Now on to the question. From journalctl(1):

-D DIR, --directory=DIR

Takes a directory path as argument. If specified, journalctl will operate on the specified journal directory DIR instead of the default runtime and system journal paths.

So, assuming that the broken system's rootfs is mounted under $DEST, use journalctl -D $DEST/var/log/journal to view its system log.

For sake of completeness: in order to view a specific file, use journalctl --file, e. g. journalctl --file /var/log/journal/$(cat /etc/machine-id)/system.journal. However, as the journal files are rotated periodically, this form is not really usable for viewing complete journals.

It's easy when you know how :)

$ systemctl list-sockets 
LISTEN                          UNIT                            ACTIVATES
...
/run/systemd/journal/dev-log    systemd-journald-dev-log.socket systemd-journald.service
/run/systemd/journal/socket     systemd-journald.socket         systemd-journald.service
/run/systemd/journal/stdout     systemd-journald.socket         systemd-journald.service
...

25 sockets listed.
Pass --all to see loaded but inactive sockets, too.

ok, I lied about it being so easy. I don't have a syslog daemon, and that means I don't have syslog.socket activated. But that's where I found the docs:

$ systemctl cat syslog.socket
# /usr/lib/systemd/system/syslog.socket
...
Documentation=man:systemd.special(7)
Documentation=http://www.freedesktop.org/wiki/Software/systemd/syslog
...
# The default syslog implementation should make syslog.service a
# symlink to itself, so that this socket activates the right actual
# syslog service.
#
# Examples:
#
# /etc/systemd/system/syslog.service -> /lib/systemd/system/rsyslog.service
# /etc/systemd/system/syslog.service -> /lib/systemd/system/syslog-ng.service
#
# Best way to achieve that is by adding this to your unit file
# (i.e. to rsyslog.service or syslog-ng.service):
#
# [Install]
# Alias=syslog.service
#
# See http://www.freedesktop.org/wiki/Software/systemd/syslog for details.

https://www.freedesktop.org/wiki/Software/systemd/syslog/ says:

Note that it is now the journal that listens on /dev/log, no longer the BSD syslog daemon directly. If your logging daemon wants to get access to all logging data then it should listen on /run/systemd/journal/syslog instead via the syslog.socket unit file that is shipped along with systemd. On a systemd system it is no longer OK to listen on /dev/log directly, and your daemon may not bind to the /run/systemd/journal/syslog socket on its own. If you do that then you will lose logging from STDOUT/STDERR of services (as well as other stuff).

Hence the answer to your question is that you're not supposed to use either of these paths with unix-dgram(). You really need specific systemd support if you want to run as a syslog daemon under it.

For an individual configuration, it sounds like you might get away with binding to /run/systemd/journal/syslog. This is definitely the least evil option, because a) it avoids fighting with journald over who owns /dev/log, b) journald will write messages to it from STDOUT/STDERR of services, which are never written to /dev/log. Given that it appears to work, I can't see any explicit disadvantages listed in the docs. The obvious disadvantage is "we do no longer recommend people to order their units after syslog.target... early boot messages are lost entirely to your implementation." There's also a warning that "many services will not be able to log to your syslog implementation", but I think that's incorrect / would only apply if you listened on /dev/log.