Docker Bridges DHCP – How to Configure Docker with Bridges and DHCP

bridgedhcpdocker

I have a lot of docker containers that I need addressable on the the same LAN as their hosts. Up until now, I have been achieving this by using setting up a bridge and manually assigning them IPs, and managing the IPs myself. An example startup would be like so:

docker run \
--net="none" \ \
--lxc-conf="lxc.network.type = veth" \
--lxc-conf="lxc.network.ipv4 = 192.168.1.3/24" \
--lxc-conf="lxc.network.ipv4.gateway = 192.168.1.254" \
--lxc-conf="lxc.network.link = br0" \
--lxc-conf="lxc.network.name = eth0" \
--lxc-conf="lxc.network.flags = up" \
-d [Docker Image ID]

With the host having the bridge defined in /etc/network/interfaces (ubuntu) like so:

auto eth0
iface eth0 inet manual

auto br0
iface br0 inet static
        address 192.168.1.2
        netmask 255.255.255.0
        gateway 192.168.1.254
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

Since I discovered serf, I have been trying to move over to using automatic discovery within the containers, so that DHCP can keep track of IPs and hand them out to containers. I since changed the startup command to:

docker run \
--net="none" \
--lxc-conf="lxc.network.type = veth" \
--lxc-conf="lxc.network.link = br0" \
--lxc-conf="lxc.network.flags = up" \
-d [Docker Image ID]
/bin/bash

and the bridge to:

auto br0
iface br0 inet dhcp
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

This resulted in the container starting up, but not having an IP. I then took advice from an online post who managed to get it done with Fedora, by calling dhclient. Unfortunately this is not working for me within ubuntu based containers.

Below are the following error messages I get under different conditions:

  • Running dhclient when I have enabled --privileged in starting the container:

    dhclient: error while loading shared libraries: libc.so.6: cannot open shared object file: Permission denied

  • Running sudo dhclient eth0 when not in --privileged

    RTNETLINK answers: Operation not permitted
mv: cannot move '/etc/resolv.conf.dhclient-new.31' to '/etc/resolv.conf': Device or resource busy

  • Running sudo dhclient or dhclient (no interface specified).

    Returns immediately and there is still no IP or network connectivity.

How can I get docker containers to grab dynamic IPs from the same subnet as their hosts, such that I can deploy containers across multiple hosts without tracking IPs?

Extra Info

  • Running DOCKER_OPTS="-e lxc" in /etc/default/docker
  • Host is Ubuntu 14.04
  • Docker containers are built using from ubuntu:14.04 in the Dockerfile.

Best Answer

It appears that this is an open issue and is this is specific to Ubuntu containers and apparmor.

A workaround was posted in there from bprodoehl:

  • Start the container as privileged with --privileged
  • Add the following line to the dockerfile: RUN mv /sbin/dhclient /usr/sbin/dhclient
  • Run dhclient eth0 and you will still see the error message: mv: cannot move '/etc/resolv.conf.dhclient-new.29' to '/etc/resolv.conf': Device or resource busy, but you will now have an IP and you can use the network.
Related Question