I have done some fairly extensive searching and can't seem to find the needle in the haystack that fixes this issue.
I have a server running Ubuntu 18.04
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
I am running LXC/LXD on the server currently with only a single container which is actually a 16.04 image. DNS works fine from within the container. I believe this eliminates any potential network issue as the problem.
In the 18.04 install the following happens when using nslookup
nslookup google.com
;; connection timed out; no servers could be reached
However when including a dns server directly as such I get a lookup to work. Again seemingly ruling out firewall/network issues
nslookup google.com 1.1.1.1
Server: 1.1.1.1
Address: 1.1.1.1#53
Non-authoritative answer:
Name: google.com
Address: 172.217.5.238
Name: google.com
Address: 2607:f8b0:4006:802::200e
As part of following tips/tricks/guides some of the following things I tried are below, as well as various outputs that might be useful in helping nail this down.
I modified the following file to look as such. I only added the name servers. I did this following one of the fixes out there.
$ cat /etc/netplan/50-cloud-init.yaml
network:
version: 2
ethernets:
ens3:
dhcp4: true
match:
macaddress: <redacted for post>
set-name: ens3
nameservers:
addresses: [8.8.4.4, 8.8.8.8, 1.1.1.1, 1.1.0.0]
This did seem to add the dns servers to the device
sudo systemd-resolve --status
Global
DNS Domain: openstacklocal
DNSSEC NTA: 10.in-addr.arpa
16.172.in-addr.arpa
168.192.in-addr.arpa
17.172.in-addr.arpa
18.172.in-addr.arpa
19.172.in-addr.arpa
20.172.in-addr.arpa
21.172.in-addr.arpa
22.172.in-addr.arpa
23.172.in-addr.arpa
24.172.in-addr.arpa
25.172.in-addr.arpa
26.172.in-addr.arpa
27.172.in-addr.arpa
28.172.in-addr.arpa
29.172.in-addr.arpa
30.172.in-addr.arpa
31.172.in-addr.arpa
corp
d.f.ip6.arpa
home
internal
intranet
lan
local
private
test
Link 5 (vethTR4JCU)
Current Scopes: none
LLMNR setting: yes
MulticastDNS setting: no
DNSSEC setting: no
DNSSEC supported: no
Link 3 (lxdbr0)
Current Scopes: none
LLMNR setting: yes
MulticastDNS setting: no
DNSSEC setting: no
DNSSEC supported: no
Link 2 (ens3)
Current Scopes: DNS
LLMNR setting: yes
MulticastDNS setting: no
DNSSEC setting: no
DNSSEC supported: no
DNS Servers: 8.8.4.4
8.8.8.8
1.1.1.1
1.1.0.0
<redacted for post>
DNS Domain: openstacklocal
Even with the dns servers listed there, no lookups are possible using either dig, or nslookup.
I did install resolvconf as part of a guide, though I think that was unnecessary and just proved to make a bigger mess.
$ ls -al /etc/resolv.conf
lrwxrwxrwx 1 root root 29 Jan 29 12:55 /etc/resolv.conf -> ../run/resolvconf/resolv.conf
cat /run/resolvconf/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53
search openstacklocal
This is as far as I seem to be able to get. If I add valid nameservers (8.8.8.8, 8.8.4.4, 1.1.1.1, etc) to /run/resolveconf/resolv.conf file:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 127.0.0.53
nameserver 8.8.8.8 # manually added in for testing
search openstacklocal
I can get lookups to work as shown below. If course as is stated in the file, these changes are overwritten on reboot.
nslookup google.com
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: google.com
Address: 172.217.15.78
Name: google.com
Address: 2607:f8b0:4004:810::200e
EDIT: output of apply command
sudo netplan --debug apply
** (generate:15710): DEBUG: 14:11:34.829: Processing input file /etc/netplan/50-cloud-init.yaml..
** (generate:15710): DEBUG: 14:11:34.830: starting new processing pass
** (generate:15710): DEBUG: 14:11:34.878: ens3: setting default backend to 1
** (generate:15710): DEBUG: 14:11:34.879: Generating output files..
** (generate:15710): DEBUG: 14:11:34.879: NetworkManager: definition ens3 is not for us (backend 1)
DEBUG:netplan generated networkd configuration exists, restarting networkd
DEBUG:no netplan generated NM configuration exists
DEBUG:ens3 not found in {}
DEBUG:Merged config:
network:
bonds: {}
bridges: {}
ethernets:
ens3:
dhcp4: true
match:
macaddress: <redacted for post>
nameservers:
addresses:
- 8.8.4.4
- 8.8.8.8
- 1.1.1.1
- 1.1.0.0
set-name: ens3
vlans: {}
wifis: {}
DEBUG:Skipping non-physical interface: lo
DEBUG:device ens3 operstate is up, not changing
DEBUG:Skipping non-physical interface: lxdbr0
DEBUG:Skipping non-physical interface: vethTR4JCU
DEBUG:{}
DEBUG:netplan triggering .link rules for lo
DEBUG:netplan triggering .link rules for ens3
DEBUG:netplan triggering .link rules for lxdbr0
DEBUG:netplan triggering .link rules for vethTR4JCU
EDIT: Requsted
sudo iptables -L -n -v
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT tcp -- lxdbr0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:53 /* generated for LXD network lxdbr0 */
0 0 ACCEPT udp -- lxdbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:53 /* generated for LXD network lxdbr0 */
0 0 ACCEPT udp -- lxdbr0 * 0.0.0.0/0 0.0.0.0/0 udp dpt:67 /* generated for LXD network lxdbr0 */
0 0 ACCEPT tcp -- ens3 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8443 /* allow connection to lxd */
2336 152K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
1 60 ACCEPT tcp -- lxdbr0 * 10.100.106.40 0.0.0.0/0 tcp dpt:22
1279 73342 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
8207 2604K ACCEPT all -- * lxdbr0 0.0.0.0/0 0.0.0.0/0 /* generated for LXD network lxdbr0 */
9496 3318K ACCEPT all -- lxdbr0 * 0.0.0.0/0 0.0.0.0/0 /* generated for LXD network lxdbr0 */
Chain OUTPUT (policy ACCEPT 70 packets, 8606 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT tcp -- * lxdbr0 0.0.0.0/0 0.0.0.0/0 tcp spt:53 /* generated for LXD network lxdbr0 */
0 0 ACCEPT udp -- * lxdbr0 0.0.0.0/0 0.0.0.0/0 udp spt:53 /* generated for LXD network lxdbr0 */
0 0 ACCEPT udp -- * lxdbr0 0.0.0.0/0 0.0.0.0/0 udp spt:67 /* generated for LXD network lxdbr0 */
Anyone know the link/solution to this problem. I'm at a loss.
Best Answer
TL;DR: allow port 53 tcp & udp to lo interface.
Even though the default policy on INPUT is ACCEPT, there is a final rule that drops anything not yet accepted. The only rules accepting traffic on port 53 are on the lxdbr0 interface. You could blanket allow everything on
lointerface or just allow ports as needed.To push a rule to allow everything on lo interface ahead of the other rules: