Systemd-nspawn container with separate IP address (network namespace) not working

networkingsystemd-nspawn

Looking at the documentation for systemd-nspawn, it must have been intended to have a very user-friendly way to launch containers in a different network namespace. You use the -n option, and simply enable systemd-networkd.service on both ends. The container gets its own IP address in one of the "private" ranges. (DNS might require some additional step).

The result is I get an IP address in the range 169.254.*.*. The default route points at the host0 interface without going through any gateway/router. Attempts to reach internet servers e.g. 8.8.8.8 fail with "No route to host". (No point working out DNS if this doesn't work).

If I run tcpdump -i ve-fedora-25 on the host, I can see the DHCP requests, but they are not responded to. systemd-networkd is definitely running on the host. The host-side logs show "Gained carrier" on ve-fedora-25, and networkctl shows this as "routable" & "configured" all in green.

My system is Fedora 25. I want an OS container I can connect to using TCP/IP, and at the same time be able to connect out to the world (e.g. to run the dnf package manager). Just as libvirt VMs work so easily out-of-the-box. What has gone wrong?

Best Answer

The problem is Fedora's firewalld. It seems nspawn was never integrated with firewalld. (nspawn isn't correctly integrated with Fedora's SELinux policy either).

As mentioned in the question, libvirt is working fine :). We can use the same trick people discovered for running containers with LXC on Fedora.

Update: the workaround stopped working after I upgraded to Fedora 30.


Run systemd-nspawn with the option --network-bridge virbr0. Instead of relying on systemd-networkd, this leverages libvirtd.service. The latter service is already started by default on Fedora. In the guest, set up your preferred DHCP client as usual.

DNS resolution when using systemd-networkd as DHCP client

Using systemd-networkd as a DHCP client might accidentally work on its own, if you may have a left-over /etc/resolv.conf from a previous container boot. But you can't rely on this working in general. It's really designed to be run together with systemd-resolved.service.

In turn, systemd-resolved is intended to be used with nss-resolve. However this is not essential AIUI.

Related Question