Ubuntu – Openconnect in network-manager does not update resolv.conf

network-manageropenconnectresolv.conf

I just updated from ubuntu 19.10 to ubuntu 20.04. Now my resolve.conf does not get updated anymore when I connect to vpn with the openconnect thing in the network manager.

This works correctly:

sudo openconnect -u user https://server

I see the resolv.conf being changed. So it's a client problem.

in /etc/NetworkManager/NetworkManager.conf I have dns=none to use the resolv.conf

/etc/resolv.conf is a file and is not symlinked to /run/systemd/resolve/resolv.conf. I did check if /run/systemd/resolve/resolv.conf was being updated by anything, but this also does not get updated.

This is from syslog:

NetworkManager[62862]: <info>  [1590658767.3686] vpn-connection[,"vpn",0]: VPN connection: (ConnectInteractive) reply received
NetworkManager[62862]: <info>  [1590658767.3714] vpn-connection[,"vpn",0]: VPN plugin: state changed: starting (3)
openconnect[63139]: Connected to somehostip:443
openconnect[63139]: SSL negotiation with somehostip
openconnect[63139]: Server certificate verify failed: signer not found
openconnect[63139]: Connected to HTTPS on somehostip 
openconnect[63139]: Got CONNECT response: HTTP/1.1 200 OK
openconnect[63139]: CSTP connected. DPD 30, Keepalive 20
openconnect[63139]: Connected as 10.0.0.2 + ipv6addresswashere, using SSL, with DTLS in progress
openconnect[63139]: Established DTLS connection (using GnuTLS). Ciphersuite (DTLS0.9)-(DHE-CUSTOM)-(AES-256-CBC)-(SHA1).
openconnect[63139]: SIOCSIFMTU: Operation not permitted
NetworkManager[62862]: <info>  [1590658768.5289] vpn-connection[,"vpn",0]: VPN connection: (IP Config Get) reply received.
NetworkManager[62862]: <info>  [1590658768.5320] vpn-connection[,"vpn",14:(vpn0)]: VPN connection: (IP4 Config Get) reply received
NetworkManager[62862]: <info>  [1590658768.5362] vpn-connection[,"vpn",14:(vpn0)]: VPN connection: (IP6 Config Get) reply received
NetworkManager[62862]: <info>  [1590658768.5373] vpn-connection[,"vpn",14:(vpn0)]: Data: VPN Gateway: somehostip
NetworkManager[62862]: <info>  [1590658768.5374] vpn-connection[,"vpn",14:(vpn0)]: Data: Tunnel Device: "vpn0"
NetworkManager[62862]: <info>  [1590658768.5374] vpn-connection[,"vpn",14:(vpn0)]: Data: IPv4 configuration:
NetworkManager[62862]: <info>  [1590658768.5375] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal Address: 10.0.0.2
NetworkManager[62862]: <info>  [1590658768.5375] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal Prefix: 19
NetworkManager[62862]: <info>  [1590658768.5375] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal Point-to-Point Address: 10.0.0.55
NetworkManager[62862]: <info>  [1590658768.5375] vpn-connection[,"vpn",14:(vpn0)]: Data:   Static Route: 0.0.0.0/0   Next Hop: 0.0.0.0
NetworkManager[62862]: <info>  [1590658768.5376] vpn-connection[,"vpn",14:(vpn0)]: Data:   Static Route: 10.0.0.0/19   Next Hop: 0.0.0.0
NetworkManager[62862]: <info>  [1590658768.5376] vpn-connection[,"vpn",14:(vpn0)]: Data:   DNS Domain: 'xxx.com'  
NetworkManager[62862]: <info>  [1590658768.5376] vpn-connection[,"vpn",14:(vpn0)]: Data: IPv6 configuration:
NetworkManager[62862]: <info>  [1590658768.5377] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal Address: ipv6addresswashere
NetworkManager[62862]: <info>  [1590658768.5377] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal Prefix: 64
NetworkManager[62862]: <info>  [1590658768.5378] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal Point-to-Point Address: ipv6addresswashere
NetworkManager[62862]: <info>  [1590658768.5378] vpn-connection[,"vpn",14:(vpn0)]: Data:   Static Route: ::/0   Next Hop: ::
NetworkManager[62862]: <info>  [1590658768.5378] vpn-connection[,"vpn",14:(vpn0)]: Data:   Static Route: ipv6addresswashere   Next Hop: ::
NetworkManager[62862]: <info>  [1590658768.5378] vpn-connection[,"vpn",14:(vpn0)]: Data:   DNS Domain: 'xxx.com'  
NetworkManager[62862]: <info>  [1590658768.5380] vpn-connection[,"vpn",14:(vpn0)]: VPN plugin: state changed: started (4)
NetworkManager[62862]: <info>  [1590658768.5534] vpn-connection[,"vpn",14:(vpn0)]: VPN connection: (IP Config Get) complete
NetworkManager[62862]: <info>  [1590658768.5548] device (vpn0): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
NetworkManager[62862]: <info>  [1590658768.5600] device (vpn0): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
NetworkManager[62862]: <info>  [1590658768.5615] device (vpn0): Activation: starting connection 'vpn0' (xxx)
NetworkManager[62862]: <info>  [1590658768.5616] manager: NetworkManager state is now CONNECTED_SITE
NetworkManager[62862]: <info>  [1590658768.5629] manager: NetworkManager state is now CONNECTED_LOCAL 
NetworkManager[62862]: <info>  [1590658768.5632] manager: NetworkManager state is now CONNECTED_SITE
NetworkManager[62862]: <info>  [1590658768.5633] policy: set 'vpn' (vpn0) as default for IPv4 routing and DNS

I looked in an older syslog from before the upgrade and there are lines in there like:

NetworkManager[1245]: <info>  [1590386910.5867] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal DNS: 10.0.0.1
NetworkManager[1245]: <info>  [1590386910.5867] vpn-connection[,"vpn",14:(vpn0)]: Data:   Internal DNS: 10.0.0.2

So those are missing now. Anyone got a clue on how to fix this?

I did try it with dns=auto to use systemd-resolve but that has the same problem. systemd-resolve –status reports no dns servers for the vpn connection, as well as on /run/systemd/resolve/resolv.conf.

Note: I anonymized all ip addresses and host names etc.

Best Answer

I found the solution. Basically, if the dns server sents ipv6 dns servers to lookup ipv4 addresses things go wrong. Openconnect will put ipv6 addresses in INTERNAL_IP4_DNS and the "network-manager-openconnect" does not expect that, treats the whole variable (and basically all dns servers) as garbage and goes on. I compiled my own network-manager-openconnect from master which has a fix for this, and that works fine.

I don't know why I have this problem after upgrading. Maybe something changed in openconnect? Or maybe in the day I upgraded my company network admin added an ipv6 dns server? (I think that is unlikely...)

If you are using network-manager-openconnect 1.2.6 or 1.2.7-dev (or maybe even a lower version, which does not contain the fix) you can compile your own version from master like so:

sudo apt-get build-dep network-manager-openconnect
mkdir ~/network-manager-openconnect_build
cd ~/network-manager-openconnect_build
git clone https://gitlab.gnome.org/GNOME/NetworkManager-openconnect.git
cd NetworkManager-openconnect
./autogen.sh
make
sudo mv /usr/lib/NetworkManager/nm-openconnect-service-openconnect-helper /usr/lib/NetworkManager/nm-openconnect-service-openconnect-helper_bak
sudo cp src/nm-openconnect-service-openconnect-helper /usr/lib/NetworkManager/nm-openconnect-service-openconnect-helper

Note: I only copied over the helper bin. In theory this could give incompatibility problems. I had no problems with it. But if you have, you might try to copy over the main bin as wel from src.

If you want, you can use the following to have extra logging for the vpn module to see in the syslog if openconnect received any dns data:

sudo nmcli general logging level KEEP domains VPN_PLUGIN:TRACE

If you see the INTERNAL_IP4_DNS being set with both ipv4 and ipv6 addresses, and you use the openconnect network manager version mentioned above, you are affected with this bug.

Related Solutions

Ubuntu – Network only using one DNS when connected to VPN

From your configuration, your dnsmasq installation is getting the list of DNS servers to use from /etc/resolv.conf. By default, dnsmasq tries to favor using DNS servers that are up, but will only send a given request to a single DNS server. This can cause problems if you have multiple DNS servers that can/will only serve certain queries.

I believe you can solve this issue by making sure you have a DNS server on your LAN (the one you use when you aren't connected to the VPN) set up in /etc/resolv.conf, as well as the DNS server on the corporate network you want to use over the VPN.

Then, you will need to edit /etc/default/dnsmasq and add or edit the DNSMASQ_OPTS= line to include --all-servers.

If you are still unable to get DNS queries with this setup, copy the resolv.conf file you created during the steps above to another location, like ~/resolv.conf, set /etc/resolv.conf up with nameserver 127.0.0.1 and set the following option in /etc/dnsmasq.conf:

resolv-file=/home/your_username/resolv.conf

That should configure your system to query your dnsmasq installation for DNS, and it will in turn use both your local DNS server and the VPN DNS server for every query.

Edit: You can find the DNS servers you are currently using for a particular connection using the nmcli tool. For finding the DNS servers used by my wireless connection, I used the following syntax:

nmcli dev list iface wlan0 | grep IP4.DNS

If you run this command while you are not connected to your VPN, and then again when you are connected and are able to resolve your corporate addresses, you should get your list of DNS servers off and on the VPN. I hope this helps.

Edit 2: Looking at your routing tables, it appears your VPN administrator has set you up to route all your traffic through the VPN while you're connected (your default gateway changes to a VPN address). Since both of your DNS servers are public addresses, and neither have a specific route set up while you are on the VPN, you are trying to do normal DNS lookups through the VPN and that is what is failing.

You may have a couple ways to make this work, depending on your VPN setup:

If the VPN will allow you to access the internet through the corporate network, but not perform DNS queries to servers on the internet, add routes to your DNS servers like so: sudo route add -host 83.255.245.11 gw 192.168.0.1, and sudo route add -host 193.150.193.150 gw 192.168.0.1 after connecting to the VPN.
If the VPN will not allow you to access the internet through the corporate network, you will need to change the default gateway settings on your computer to point at 192.168.0.1 after you connect to the VPN. In this case, you will want to set up your usual default gateway and then add network routes to access VPN-only equipment.

You may need to whittle down your routing table in the connected-to-the-VPN case shown in your second pastebin to the following:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG    0      0        0 eth0
192.168.0.0     0.0.0.0         255.255.255.0   U     1      0        0 eth0
10.100.0.0      10.100.0.105    255.255.255.0   UGH   0      0        0 tun0
10.100.0.105    0.0.0.0         255.255.255.255 UH    0      0        0 tun0

Then, add routes as you need to in order to access the corporate equipment. In the routing table shown above I have assumed a /24 network on the VPN, which may be incorrect. You'll have to set the mask appropriately.

Ubuntu – Getting openconnect vpn to work through network-manager

Check if there is a mismatch between the host you are trying to resolve via the VPN and the "DNS Domain" that the Cisco VPN server is sending.

To check for this, open a terminal and run:

tail -f /var/log/syslog

Then start openconnect via network manager. You'll see a whole bunch of output come through, including some lines like this:

Dec 5 12:54:31 canoe NetworkManager[1266]: Internal DNS: 10.0.20.21

Dec 5 12:54:31 canoe NetworkManager[1266]: Internal DNS: 10.10.3.32

Dec 5 12:54:31 canoe NetworkManager[1266]: DNS Domain: 'internal.example.com'

And further down you'll see

Dec 5 12:54:31 canoe dnsmasq[1871]: using nameserver 10.0.20.21#53 for domain internal.example.com

This means that the VPN server is instructing the client that the DNS servers should be used to resolve hosts within internal.example.com, such as server.internal.example.com.

In my case, I need to resolve server.example.com (and was not getting any result).

The solution for me was to go into the VPN settings and add example.com as an additional search domain:

Don't forget to disconnect VPN and then re-connect for the setting to take effect.

Best Answer

Related Solutions

Ubuntu – Network only using one DNS when connected to VPN

Ubuntu – Getting openconnect vpn to work through network-manager

Related Question