How to allow local LAN access while connected to Cisco VPN

The problem with Anyconnect is that it first modifies the routing table, then babysits it and fixes it up should you modify it manually. I found a workaround for this. Works with version 3.1.00495, 3.1.05152, 3.1.05170, and probably anything else in the 3.1 family. May work with other versions, at least similar idea should work assuming the code does not get rewritten. Fortunately for us Cisco has put the babysitter "baby is awake" call into a shared library. So the idea is that we prevent action by vpnagentd via LD_PRELOAD.

1. First we create a file hack.c:

   #include <sys/socket.h>
   #include <linux/netlink.h>
   
   int _ZN27CInterfaceRouteMonitorLinux20routeCallbackHandlerEv()
   {
     int fd=50;          // max fd to try
     char buf[8192];
     struct sockaddr_nl sa;
     socklen_t len = sizeof(sa);
   
     while (fd) {
        if (!getsockname(fd, (struct sockaddr *)&sa, &len)) {
           if (sa.nl_family == AF_NETLINK) {
              ssize_t n = recv(fd, buf, sizeof(buf), MSG_DONTWAIT);
           }
        }
        fd--;
     }
     return 0;
   }
   

Note: This code works only with Linux. For applying this solution to a macOS machine, see the macOS adapted version.

2. Then compile it like this:

   gcc -o libhack.so -shared -fPIC hack.c
   

3. Install libhack.so into the Cisco library path:

   sudo cp libhack.so  /opt/cisco/anyconnect/lib/
   

4. Bring down the agent:

   /etc/init.d/vpnagentd stop
   

5. Make sure it really is down

   ps auxw | grep vpnagentd
   

   If not, kill -9 just to be sure.

6. If you have /etc/init.d/vpnagentd, then fix it up by adding LD_PRELOAD=/opt/cisco/anyconnect/lib/libhack.so where the underlying executable is being invoked so it looks like this:

   LD_PRELOAD=/opt/cisco/anyconnect/lib/libhack.so /opt/cisco/anyconnect/bin/vpnagentd
   

   More modern AnyConnect installations eschew /etc/init.d/vpnagentd in favor of /lib/systemd/system/vpnagentd.service, in which case you'll want something like:

   sudo mv /opt/cisco/anyconnect/bin/vpnagentd /opt/cisco/anyconnect/bin/vpnagentd.orig &&
   { echo '#!/bin/bash' &&
     echo "LD_PRELOAD=$HOME/vpn/libhack.so exec /opt/cisco/anyconnect/bin/vpnagentd.orig"
   } | sudo tee /opt/cisco/anyconnect/bin/vpnagentd &&
   sudo chmod +x /opt/cisco/anyconnect/bin/vpnagentd
   

7. Now start the agent:

   /etc/init.d/vpnagentd start
   

8. Fix up iptables, because AnyConnect messes with them:

   iptables-save | grep -v DROP | iptables-restore
   

   You may want to do something more advanced here to allow access only to certain LAN hosts.

9. Now fix up the routes as you please, for example:

   route add -net 192.168.1.0 netmask 255.255.255.0 dev wlan0
   

10. Check to see if they are really there:

    route -n

A previous, simpler version of this hack gave a function that only did "return 0;" - that poster noted that "The only side effect that I've observed so far is that vpnagentd is using 100% of CPU as reported by top, but overall CPU is only 3% user and 20% system, and the system is perfectly responsive. I straced it, it seems to be doing two selects in a loop when idle returning from both quickly, but it never reads or writes - I suppose the call that I cut out with LD_PRELOAD was supposed to read. There might be a cleaner way to do it, but it is good enough for me so far. If somebody has a better solution, please share."

The problem with the trivial hack is it caused a single cpu core to be 100% all the time, effectively reducing your hardware cpu thread count by one - whether your vpn connection was active or not. I noticed that the selects the code was doing were on a netlink socket, which sends vpnagentd data when the routing table changes. vpnagentd keeps noticing there's a new message on the netlink socket and calls the routeCallBackHandler to deal with it, but since the trivial hack doesn't clear the new message it just keeps getting called again and again. the new code provided above flushes the netlink data so the endless loop which caused the 100% cpu doesn't happen.

If something does not work, do gdb -p $(pidof vpnagentd), once attached:

b socket
c
bt

and see which call you are in. Then just guess which one you want to cut out, add it to hack.c and recompile.