Summary: Is there a way to make sure that the NFS server isn't started by systemd until it can properly resolve client machine names specified in /etc/exports
?
Problem description:
I have found that NFS shares are not made properly available after the server (running 16.10) reboots. Clients get "access denied by server" errors, until exportfs -ra
or service nfs-server restart
is manually run on the server. After that, everything works as expected.
The server's /etc/exports
contains only:
/mnt/raidarray clientmachine(rw)
where clientmachine
is the hostname of the NFS client machine on the local network.
Problem identification: The output of systemctl status nfs-server
(below) makes the problem clear: the name of the client can't be resolved at the time that the NFS server was started.
● nfs-server.service - NFS server and services
Loaded: loaded (/lib/systemd/system/nfs-server.service; enabled
Active: active (exited) since Tue 2017-01-17 16:47:38 CST; 26min ago
Main PID: 1520 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 4915)
CGroup: /system.slice/nfs-server.service
Jan 17 16:47:38 servermachine exportfs[1511]: exportfs: Failed to resolve clientmachine
Jan 17 16:47:38 servermachine systemd[1]: Started NFS server and services.
NetworkManager-wait-online.service
is enabled, which I had understood serves to make sure that network.target
(on which nfs-server
depends) is not satisfied until network-online.target
is satisfied.
Things that did not help:
-
In case I misunderstood what
NetworkManager-wait-online.service
does, I tried adding an explicitAfter=network-online.target
andWants=network-online.target
tonfs-server.service
. This doesn't fix anything. I see the new dependency show up insystemctl list-dependencies
, but name resolution still fails at boot. So it seems thatnetwork-online.target
doesn't guarantee that hostname resolution can happen. -
Some googling suggested that requiring
nss-lookup.target
would ensure that network resolution is available, but adding it as aWants
and/orAfter
dependency tonfs-server.service
doesn't fix things either! -
Adding a
Wants
and/orAfter
dependency onsystemd-resolved.service
instead ofnss-lookup.target
doesn't fix things either.
After adding all of these dependencies, nfs-server
starts up very very late in the boot process (just before the desktop login things), but it still can't resolve hosts. Based on systemd-analyze plot
, it appears that nmbd
is wedged around this time, but I don't know if that's related.
Configuration information: This is on a desktop version of kubuntu 16.10 that is basically a fresh install.
NetworkManager.service
is enabled and systemd-networkd.service
is disabled — I didn't change that from the default.
Here is the NetworkManager-wait-online
service definition:
[Unit]
Description=Network Manager Wait Online
Documentation=man:nm-online(1)
Requisite=NetworkManager.service
After=NetworkManager.service
Before=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/bin/nm-online -s -q --timeout=30
RemainAfterExit=yes
[Install]
WantedBy=network-online.target
As a workaround I could hard-code IP addresses instead of hostnames into /etc/exports
or /etc/hosts
, but this seems somewhat brittle. Is there a better answer?
EDIT – Update: following @muru's advice below, I tried making a script to wait to resolve hostnames — just to see how long it takes. My script has to wait tens of seconds after systemd-resolved
starts before it can actually resolve a host. This is very bizarre. I wonder if it's actually a local networking issue? Sounds like maybe a hard-coded (and auto-updating, perhaps, as suggested by @mark-stosberg) /etc/exports
or /etc/hosts
file is warranted.
Best Answer
In terms of what you are trying to do with
systemd
, you've already tried to do the best thing, which is to set your service to start afternetwork-online
. I read a lot of asystemd
questions, and others have reported problems with the network not being fully online when they do this.I recommend the idea of putting IP addresses in
/etc/exports
or/etc/hosts
as you recommend. This simplifies an important disk mounting task during boot-up.To make the system more robust, you could use a cron job or systemd timer that checks periodically to see if the DNS for the interesting hosts have changed. If so, automatically update
/etc/hosts
with the new values.