Two local networks: routing confusion

airportethernetNetworkwifi

On my 17" MBP (running Mavericks) I'm sometimes signed on to two different LANs, one with my AirPort interface and one with Ethernet. My Networks are set up (in the Network System Preference Panel) such that the Ethernet service is higher priority than the AirPort.

Sometimes (not always), when I try to connect to an IP address on the network I'm connected to via AirPort, it (I assume) tries to connect to the IP via the Ethernet network and eventually times out. Pinging an IP on the AirPort network also fails.

I set up the Ethernet network as higher priority than the AirPort network, so it's working as intended, you say – 'what's the problem'? Well, the IP I'm trying to connect to is a local IP (e.g. 192.168.x.x), while the Ethernet router's IP is like 10.0.0.1, and it's assigning IP addresses using DHCP like 10.0.0.2, 10.0.0.3, etc. So if I try to connect to say 192.168.1.2, why is it trying to go over Ethernet, in 10.0.0.x land? How do I have to set up the networks so that the laptop "realizes" it needs to use the AirPort interface for connecting to 192.168.x.x addresses? BTW, reversing the service order just makes it so I have the same problem in reverse – suddenly all connections to Ethernet IPs time out.

Edit:

I'm providing some concrete info below to help diagnose the problem. Today, for no apparent reason, the situation is reversed – my Ethernet network, which routes to the Internet, is working fine, but packets to my AirPort network (10.11.12.*) are being incorrectly routed through the Ethernet network (192.168.20.*). I wonder if this has to do with the order in which I connected to the networks when I got to work this morning.

Notice that it attempts to route packets to 10.11.12.13 via 192.168.20.1 which is the "default-er" gateway (above 10.11.12.13). I have full control of both routers, so I can change their configuration as necessary to keep the two networks from becoming confused on my MBP. I just have no idea what I need to do to accomplish that.

$ netstat -nr
Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            192.168.20.1       UGSc            8        0     en0
default            10.11.12.13        UGScI           0        0     en1
10.11.12/24        link#4             UCSI            2        0     en1
10.11.12.13        de:ad:be:ef:0:0    UHLWIi          1      174     en1    976
10.11.12.14        127.0.0.1          UHS             1        0     lo0
10.11.12.255       ff:ff:ff:ff:ff:ff  UHLWbI          0        3     en1
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              1    62540     lo0
169.254            link#4             UCS             0        0     en1
192.168.20         link#5             UCS             6        0     en0
192.168.20.1       48:5b:39:e8:b:8    UHLWIir         9      171     en0   1198
192.168.20.128     0:e0:4d:bf:2:2b    UHLWIi          1      109     en0   1077
192.168.20.141     68:9:27:5c:f1:e9   UHLWI           0        0     en0   1174
192.168.20.143     127.0.0.1          UHS             0        0     lo0
192.168.20.148     a8:88:8:c9:40:a4   UHLWI           0        0     en0   1021
192.168.20.255     ff:ff:ff:ff:ff:ff  UHLWbI          0        4     en0


$ ping 10.11.12.13
PING 10.11.12.13 (10.11.12.13): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
^C


$ route -n get 10.11.12.13
   route to: 10.11.12.13
destination: default
       mask: default
    gateway: 192.168.20.1
  interface: en0
      flags: <UP,GATEWAY,DONE,STATIC,PRCLONING>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1500         0 

Edit 2: Traceroute

$ traceroute 10.11.12.13
traceroute to 10.11.12.13 (10.11.12.13), 64 hops max, 52 byte packets
 1  unknown (192.168.20.1)  0.655 ms  0.474 ms  0.365 ms
 2  10.253.0.1 (10.253.0.1)  7.806 ms  7.713 ms  7.619 ms
 3  173-219-249-22-link.sta.suddenlink.net (173.219.249.22)  9.155 ms  9.189 ms  7.729 ms
 4  173-219-249-248-link.sta.suddenlink.net (173.219.249.248)  23.200 ms  24.235 ms  23.603 ms
 5  173-219-229-77-link.sta.suddenlink.net (173.219.229.77)  821.529 ms  778.116 ms  772.398 ms
 6  173-219-229-74-link.sta.suddenlink.net (173.219.229.74)  21.760 ms  23.201 ms
    173-219-229-98-link.sta.suddenlink.net (173.219.229.98)  20.502 ms
 7  173-219-229-77-link.sta.suddenlink.net (173.219.229.77)  890.499 ms  884.132 ms  906.584 ms
 8  173-219-229-80-link.sta.suddenlink.net (173.219.229.80)  21.744 ms
    173-219-229-98-link.sta.suddenlink.net (173.219.229.98)  22.142 ms  23.222 ms
 9  173-219-229-77-link.sta.suddenlink.net (173.219.229.77)  904.088 ms^C

Best Answer

Just nailed this. The non-Internet-connected network (the one I usually access via AirPort) had its router set up incorrectly. I got a way out from the Unix guys here: https://unix.stackexchange.com/a/113622/21486

Basically if you set up a hostapd wireless access point on a Linux machine and you're using DHCP, you need to make sure you don't have an option routers line in your dhcpd.conf. That line will make the router advertise to clients a spurious default gateway which, depending on the alignment of the planets, will blackhole your traffic intended for the other network.