Namespace management with ip netns (iproute2)

dnsmasqhostapdiproutenamespace

I would like to know where to look up changes applied to custom namespaces. Let's say I have a namespace "ns1" with virtual interfaces "vpeer" and "lo" inside and I would like to modify configurations of interface "vpeer" via dnsmasq or hostapd then I know that I am able to do so with e.g.

$ ip netns exec ns1 bash
$ dnsmasq --interface=vpeer-ns1 --dhcp-range=192.168.0.100,192.168.0.200,255.255.255.0

and netns is doing the rest of the job by mounting it for me and manage any assignments, which I am thankful for. However, in global namespace I could monitor changes made in corresponding files or apply them directly in a file. For example in /etc/dnsmasq.conf or /etc/hostapd/hostapd.conf.

So my question is, how can I copy default namespace behaviour within a certain netns namespace. I really need to know what I have already done and what is the current configuration of the interfaces in "ns1". Not least because most tutorials explain procedures like this. The ip netns man page says that configuration files are located in

$ /etc/netns/ns1/

but not even the directory /etc/netns does exist after creation of namespace "ns1". The new namespace is only represented in /var/run/netns/ns1. This file is always empty – even after applying dnsmasq commands as above – and nano shows up with the empty file but gives me some strange error afterwards when file is closed saying that

ns1: Argument is invalid.

So I switched to namespace "ns1" and tried again. Same result. So I turned every device in the new namespace down with

$ ip netns exec ns1 ip link set dev lo down
$ ip netns exec ns1 ip link set dev vpeer down

Again, same result.

So how can I access configrations files with respect to a given namespace, since afaik the applications are not aware of the non-default namespace at all? Thank you in advance.

Best Answer

You have two questions in one:

what are the empty files in /var/run/netns/?
how to use /etc/netns/ which doesn't even exist?

What are the empty files in `/var/run/netns/`?

Some understanding of namespaces is needed. I can't explain it in detail here. Suffice to say that a namespace once created will (partition some kernel resource such as network, and) exist as long as something is using it. Mostly two thing will use it: processes, or mountpoints in the special pseudo-filesystem nsfs. Once they stop using it or disappear, the namespace also disappears.

ip netns (which specializes in network namespaces) chooses to keep network namespaces by mounting a reference to them. Those references are all avalable in /proc/<pid>/net/ as symlinks. Example:

$ ls -l /proc/$$/ns/net
lrwxrwxrwx. 1 user user 0 Sep 25 01:05 /proc/19120/ns/net -> net:[4026531992]
$ stat -L -f -c %T /proc/19120/ns/net # filesystem type. "df" on this would be confused and point to the wrong mountpoint anyway.
nsfs
$ stat -L -c %i /proc/19120/ns/net # inode
4026531992

ip netns add just bind-mounts them in /var/run/netns/<NAME>. Note that while you usually bind-mount a directory over a directory, you can also bind-mount a file over a file. Note also that this file is a pseudo-file in the nsfs filesystem. It's not designed to be read, and can't be. It's mostly used to be passed around as a reference, by opening (but not reading or writing) it or mounting it:

# strace -e unshare,mount  ip netns add ns1
mount("", "/var/run/netns", 0x5625f6ca80e3, MS_REC|MS_SHARED, NULL) = 0
unshare(CLONE_NEWNET)                   = 0
mount("/proc/self/ns/net", "/var/run/netns/ns1", 0x5625f6ca80e3, MS_BIND, NULL) = 0
+++ exited with 0 +++
# ip netns list
ns1
# ls -l /var/run/netns/ns1
-r--r--r--. 1 root root 0 Sep 25 01:08 /var/run/netns/ns1
# cat /var/run/netns/ns1
cat: /var/run/netns/ns1: Invalid argument
# stat -f -c %T /var/run/netns/ns1
nsfs
# stat -c %i /var/run/netns/ns1
4026532824
# ip netns exec ns1 sleep 99 &
[1] 19620
# ls -l /proc/19620/ns/net
lrwxrwxrwx. 1 root root 0 Sep 25 01:23 /proc/19620/ns/net -> net:[4026532824]

So sleep 99's network namespace is the one from the mountpoint /var/run/netns/ns1. With this method, for example, it's possible to have a network namespace with a bridge linked to other namespaces with veth interfaces and not running any process, because none are needed.

You can manually delete namespaces created by ip netns with:

# umount /var/run/netns/ns1 
# rm /var/run/netns/ns1
# ip netns list
#

Of course the network namespace will really only disappear once the last reference using it (eg: process) will also disappear. You could even put it back, if the previous sleep command is still running:

# touch /var/run/netns/ns1
# mount --bind /proc/19620/ns/net /var/run/netns/ns1
# ip netns
ns1

This can be used to "copy" the network namespace's part of a full container, you just need to know some pid on the host side (eg: lxc-info -Hp -n container or docker inspect --format '{{.State.Pid}}' container) and mount its network namespace. Handy to run debug commands not available inside the container (tcpdump ...).

Everything above is ephemeral, eg it disappears after a reboot.

How to use `/etc/netns/` which doesn't even exist?

The ip netns manpage tells (bold emphasis mine):

For applications that are aware of network namespaces, the convention is to look for global network configuration files first in /etc/netns/NAME/ then in /etc/. For example, if you want a different version of /etc/resolv.conf for a network namespace used to isolate your vpn you would name it /etc/netns/myvpn/resolv.conf.

Now, is dnsmasq "aware" of network namespaces? If it was (with ip netns's definition) it would check for /etc/netns/ somehow. There's no mention of "netns" in its manpage. The current Debian's (similar to Raspberry's) source doesn't include the word netns anywhere: it's not network namespace aware.

That's why ip netns manpage's following sentence tells:

ip netns exec automates handling of this configuration, file convention for network namespace unaware applications, by creating a mount namespace and bind mounting all of the per network namespace configure files into their traditional location in /etc.

It's not really explained, but together with the previous sentence this really means that ip netns exec will bind-mount, if both the source and target exist, /etc/netns/<NAME>/whatever on /etc/whatever. /etc/netns and the chosen network namespace names inside, which will contain the configurations have to be created first, manually, and actual configuration content probably copied inside and edited: ip netns exec will use them if found, but not create them.

Example:

# mkdir -p /etc/netns/ns1
# echo something > /etc/netns/ns1/whatever
# : > /etc/whatever
# ip netns add ns1
# cat /etc/whatever
# ip netns exec ns1 cat /etc/whatever
something

What really happened behind:

# strace -e open,unshare,setns,mount,umount2 ip netns exec ns1 cat /etc/whatever 2>&1 |egrep '^(open.*/etc/netns|unshare|setns|mount|umount|something)'
setns(5, CLONE_NEWNET)                  = 0
unshare(CLONE_NEWNS)                    = 0
mount("", "/", 0x55947eb550e3, MS_REC|MS_SLAVE, NULL) = 0
umount2("/sys", MNT_DETACH)             = 0
mount("ns1", "/sys", "sysfs", 0, NULL)  = 0
open("/etc/netns/ns1", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 5
mount("/etc/netns/ns1/whatever", "/etc/whatever", 0x55947eb550e3, MS_BIND, NULL) = 0
something

Of course this works the same for directories inside /etc/netns/ns1 instead of files: the whole directory will appear directly in /etc/ masking the host's directory of the same name, only if it exists.

So if dnsmasq's default configuration unless told otherwise is /etc/dnsmasq.conf, You can create an alternate configuration for ns1 as /etc/netns/ns1/dnsmasq.conf and ip netns exec will make it appear as /etc/dnsmasq.conf to dnsmasq, if there's also already a /etc/dnsmasq.conf file, even if it was empty. Since it's in /etc this isn't ephemeral and will be reused whenever you're recreating the same network namspace name. Same for hostapd and the whole /etc/hostapd/ directory which can be copied as /etc/netns/ns1/hostapd/ and its files then modified inside. Be careful with symlinks pointing outside. You can now run the commands as usual inside ip netns exec ns1: they'll use the new configuration file.

Related Solutions

Linux – Trying to run OpenVPN in Network Namespace

Starting openvpn inside the network namespace is safer. I use the following script (fork of Schnouki's) to create namespace, configure firewall, DNS, test connectivity, start openvpn, and finally start a torrent client. I put TODOs in the script where you have to adjust it to your needs.

#!/bin/sh
# start openvpn tunnel and torrent client inside Linux network namespace
#
# this is a fork of schnouki's script, see original blog post
# https://schnouki.net/posts/2014/12/12/openvpn-for-a-single-application-on-linux/
#
# original script can be found here
# https://gist.github.com/Schnouki/fd171bcb2d8c556e8fdf

# ------------ adjust values below ------------
# network namespace
NS_NAME=myVPN
NS_EXEC="ip netns exec $NS_NAME"
# user for starting the torrent client
REGULAR_USER=heinzwurst
# ---------------------------------------------

# exit on unbound variable
set -u

# exit on error
set -e
set -o pipefail

# trace option
#set -x

if [ $USER != "root" ]; then
    echo "This must be run as root."
    exit 1
fi

start_vpn() {
    echo "Add network interface"

    # Create the network namespace
    ip netns add $NS_NAME

    # Start the loopback interface in the namespace
    $NS_EXEC ip addr add 127.0.0.1/8 dev lo
    $NS_EXEC ip link set lo up

    # Create virtual network interfaces that will let OpenVPN (in the
    # namespace) access the real network, and configure the interface in the
    # namespace (vpn1) to use the interface out of the namespace (vpn0) as its
    # default gateway
    ip link add vpn0 type veth peer name vpn1
    ip link set vpn0 up
    ip link set vpn1 netns $NS_NAME up

    ip addr add 10.200.200.1/24 dev vpn0
    $NS_EXEC ip addr add 10.200.200.2/24 dev vpn1
    $NS_EXEC ip link set dev vpn1 mtu 1492
    $NS_EXEC ip route add default via 10.200.200.1 dev vpn1

    # Configure the nameserver to use inside the namespace
    # TODO use VPN-provided DNS servers in order to prevent leaks
    mkdir -p /etc/netns/$NS_NAME
    cat >/etc/netns/$NS_NAME/resolv.conf <<EOF || exit 1
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF

    # IPv4 NAT, you may need to adjust the interface name prefixes 'eth' 'wlan'
    iptables -t nat -A POSTROUTING -o eth+ -m mark --mark 0x29a -j MASQUERADE
    iptables -t nat -A POSTROUTING -o wlan+ -m mark --mark 0x29a -j MASQUERADE
    iptables -t mangle -A PREROUTING -i vpn0 -j MARK --set-xmark 0x29a/0xffffffff

    # TODO create firewall rules for your specific application (torrent)
    # or just comment the line below
    $NS_EXEC iptables-restore < /etc/iptables/iptables-$NS_NAME.rules

    # we should have full network access in the namespace
    $NS_EXEC ping -c 3 www.google.com

    # start OpenVPN in the namespace
    echo "Starting VPN"
    cd /etc/openvpn
    # TODO create openvpn configuration in /etc/openvpn/$NS_NAME.conf
    $NS_EXEC openvpn --config $NS_NAME.conf &

    # wait for the tunnel interface to come up
    while ! $NS_EXEC ip link show dev tun0 >/dev/null 2>&1 ; do sleep .5 ; done
}

stop_vpn() {
    echo "Stopping VPN"
    ip netns pids $NS_NAME | xargs -rd'\n' kill
    # TODO wait for terminate

    # clear NAT
    iptables -t nat -D POSTROUTING -o eth+ -m mark --mark 0x29a -j MASQUERADE
    iptables -t nat -D POSTROUTING -o wlan+ -m mark --mark 0x29a -j MASQUERADE
    iptables -t mangle -D PREROUTING -i vpn0 -j MARK --set-xmark 0x29a/0xffffffff

    echo "Delete network interface"
    rm -rf /etc/netns/$NS_NAME

    ip netns delete $NS_NAME
    ip link delete vpn0
}

# stop VPN on exit (even when error occured)
trap stop_vpn EXIT

start_vpn

# TODO start your favorite torrent client
$NS_EXEC sudo -u $REGULAR_USER transmission-gtk

Why does `systemd-nspawn -n` network namespace not show in `ip netns list`

Both systemd-nspawn and ip-netns use namespaces, specifically network namespaces. The difference, as explained in the ip-netns manual, is that ip-netns deals with named network namespaces.

By convention a named network namespace is an object at /var/run/netns/NAME that can be opened. The file descriptor resulting from opening /var/run/netns/NAME refers to the specified network namespace. Holding that file descriptor open keeps the network namespace alive.

Anonymous network namespaces

The namespaces(7) manual explains that in general, a namespace is an abstraction associated with the lifetime of the processes in it:

Each process has a /proc/[pid]/ns/ subdirectory containing one entry for each namespace that supports being manipulated by setns(2) ... Opening one of the files in this directory (or a file that is bind mounted to one of these files) returns a file handle for the corresponding namespace of the process specified by pid. As long as this file descriptor remains open, the namespace will remain alive, even if all processes in the namespace terminate.

On my system, the most recently launched systemd process (pgrep -f -n systemd\$) is the init process of a container started using the default systemd-nspawn@.service template unit, which enables --network-veth and thus --private-network (it also adds --private-users). This command shows that the container's anonymous network namespace is different to the root network namespace, and owned by the container's root user:

# ls -l /proc/1/ns/net /proc/$(pgrep -f -n systemd\$)/ns/net
lrwxrwxrwx 0 root           /proc/1/ns/net -> net:[4026532008]
lrwxrwxrwx 0 vu-container-0 /proc/700/ns/net -> net:[4026532656]

This anonymous network namespace disappears when the container is terminated. However, if I want to make it a named network namespace that can be managed with ip-netns during the life of the container, I can bind mount it under /run/netns:

# mount --bind /proc/$(pgrep -f -n systemd\$)/ns/net /run/netns/container
# ip netns list
container (id: 1)

Creating named network namespaces with systemd

You've also pointed out systemd-nspawn's --network-namespace-path option, which is equivalent to the NetworkNamespacePath= setting documented in systemd.unit(5). It can only assign containers and units to a network namespace that already exists. Because a process can only be in one namespace, --network-namespace-path is incompatible with options like --private-network which create an anonymous network namespace and isolate the container in it.

It seems that systemd will get a Namespace= setting in some future release of systemd after v246 (v245 was released in March 2020). This will allow units to create their own named network namespaces, rather than being assigned to an existing namespace with NetworkNamespacePath= or creating a new anonymous namespace with PrivateNetwork=. When this feature is merged, it would make sense for Namespace=%i to be added to the systemd-nspawn@.service template, so that containers' network namespaces are named by default.