OK, I have a very interesting case. I have a custom DNS server that is vending out IP addresses to varying servers in my test network. If I use dig, nslookup, or host, it tells me that it's properly resolving the IPs. If I ping or curl the hostname, it resolves to the actual external internet IP.
In my Network settings, I have only my custom DNS server, and this is correctly reflected in /etc/resolv.conf. I do not have this host in my /etc/hosts. If I add them, it will work, but I have multiple hosts that I'd have to add and remove, and I really want to know why this isn't working.
I've tried dumping the DNS cache using both dscacheutil and restarting mDNSResponder. This has had no effect. If I lookup the hostname via dscacheutil, it resolves incorrectly.
Clearly, Directory Services is messed up somehow. I've opened up Directory Utility, but I'm not too sure what to look for there.
Running dscacheutil -configuration shows:
azkaban:dnstest kylar$ dscacheutil -configuration
DirectoryService Cache search policy:
/Local/Default
Unable to get details from the cache node
Unable to get cache configuration information
Any suggestions as to what to look for next? Also note that I'm using fully qualified domain names, i.e., foo.example.com or www.example.com.
UPDATE:
I also have a VPN enabled, but it's second in the Network list. The VPN is vending me a "lookup domain" that is the same as the domain that I'm trying to lookup. Somehow it's vending me a DNS server that is overriding the DNS server that I have set (Looks like a case of most-specific lookup). How can I tell it NOT to look via that server?
Best Answer
And I found it. The VPN config was auto-configuring it's own DNS servers, which were overriding my setup, because the VPN was also telling it to look up hosts in the domain that I was overriding. This caused the Directory Services to use the VPN vended DNS servers.
To fix, I added my custom DNS server as the first lookup entry under the VPN DNS config, and that worked.