dig +trace
works by pretending it's a name server and works down the namespace tree using iterative queries starting at the root of the tree, following referrals along the way.
The first thing it does is ask the normal system DNS server for NS records for "."
After it gets a response, which will be the current list of root name servers, it'll pick one and then ask for the A record for that name if it didn't get it in the additional records section the first time so it's got an IP address to send the next query to. Let's say it picks f.root-servers.net whose IP address is 192.5.5.241.
At this point, let's use dig +trace www.google.co.uk.
as our command with a domain name we want to trace the resolution path for.
The next behind-the-scenes query will be this:
$ dig +norecurse @192.5.5.241 www.google.co.uk
; <<>> DiG 9.9.4 <<>> +norecurse @192.5.5.241 www.google.co.uk
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8962
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 11, ADDITIONAL: 15
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.co.uk. IN A
;; AUTHORITY SECTION:
uk. 172800 IN NS ns5.nic.uk.
uk. 172800 IN NS ns6.nic.uk.
uk. 172800 IN NS ns4.nic.uk.
uk. 172800 IN NS nsc.nic.uk.
uk. 172800 IN NS ns2.nic.uk.
uk. 172800 IN NS ns3.nic.uk.
uk. 172800 IN NS nsd.nic.uk.
uk. 172800 IN NS nsa.nic.uk.
uk. 172800 IN NS ns7.nic.uk.
uk. 172800 IN NS nsb.nic.uk.
uk. 172800 IN NS ns1.nic.uk.
;; ADDITIONAL SECTION:
ns1.nic.uk. 172800 IN A 195.66.240.130
ns2.nic.uk. 172800 IN A 217.79.164.131
ns3.nic.uk. 172800 IN A 213.219.13.131
ns4.nic.uk. 172800 IN A 194.83.244.131
ns5.nic.uk. 172800 IN A 213.246.167.131
ns6.nic.uk. 172800 IN A 213.248.254.130
ns7.nic.uk. 172800 IN A 212.121.40.130
nsa.nic.uk. 172800 IN A 156.154.100.3
nsb.nic.uk. 172800 IN A 156.154.101.3
nsc.nic.uk. 172800 IN A 156.154.102.3
nsd.nic.uk. 172800 IN A 156.154.103.3
ns1.nic.uk. 172800 IN AAAA 2a01:40:1001:35::2
ns4.nic.uk. 172800 IN AAAA 2001:630:181:35::83
nsa.nic.uk. 172800 IN AAAA 2001:502:ad09::3
;; Query time: 45 msec
;; SERVER: 192.5.5.241#53(192.5.5.241)
;; WHEN: Tue Feb 11 19:19:14 MST 2014
;; MSG SIZE rcvd: 507
Wow, so now we know that there are nameservers for uk
and that's the only thing the root server knows about. This is a referral, because we did not ask for recursion (+norecurse
turns it off).
Great, we rinse and repeat. This time we pick one of the uk
nameservers and ask it the same question.
$ dig +norecurse @195.66.240.130 www.google.co.uk
; <<>> DiG 9.9.4 <<>> +norecurse @195.66.240.130 www.google.co.uk
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 618
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;www.google.co.uk. IN A
;; AUTHORITY SECTION:
google.co.uk. 172800 IN NS ns1.google.com.
google.co.uk. 172800 IN NS ns3.google.com.
google.co.uk. 172800 IN NS ns2.google.com.
google.co.uk. 172800 IN NS ns4.google.com.
;; Query time: 354 msec
;; SERVER: 195.66.240.130#53(195.66.240.130)
;; WHEN: Tue Feb 11 19:22:47 MST 2014
;; MSG SIZE rcvd: 127
Awesome, now we have found out that the uk
top-level nameserver knows there's a zone called google.co.uk
and tells us to go ask those nameservers our question. This is another referral.
Rinse, repeat.
This time, however, we didn't get A records in the additional records section of the response, so we pick one, say ns2.google.com, and we have to go find it's address. We restart a query (at the root again) and chase down the tree to find the IP address for ns2.google.com. I'll skip that part for brevity, but we learn that the IP for it is 216.239.34.10.
So our next query is:
$ dig +norecurse @216.239.34.10 www.google.co.uk
; <<>> DiG 9.9.4 <<>> +norecurse @216.239.34.10 www.google.co.uk
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33404
;; flags: qr aa; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.google.co.uk. IN A
;; ANSWER SECTION:
www.google.co.uk. 300 IN A 74.125.225.216
www.google.co.uk. 300 IN A 74.125.225.223
www.google.co.uk. 300 IN A 74.125.225.215
;; Query time: 207 msec
;; SERVER: 216.239.34.10#53(216.239.34.10)
;; WHEN: Tue Feb 11 19:26:43 MST 2014
;; MSG SIZE rcvd: 82
And we're done! (finally) How do we know we're done? We got an answer to our query, which was the A records for www.google.co.uk. You can also tell because it's not a referral anymore, the aa
bit is set in the last response meaning this is the authoritative answer for your query.
So that's what's happening each step along the way when you use dig +trace
.
Note that if you have a DNSSEC-aware version of dig and you add +dnssec
to the command, you may see a bunch more records. What those extra records are is left as an exercise for the reader... but gets into how dig +sigchase
works.
To be short, Both of the queries you have mentioned in the question are non-authoritative query.
DNS records for a domain can be queried from a caching DNS server or from an authoritative DNS server. So when you want to query a caching DNS server you can either specify the DNS IP address or if not specified, the default DNS server that has been configured in /etc/resolv.conf
will be taken.
Non authoritative query
$ dig stackexchange.com
or
$ dig stackexchange.com @8.8.8.8
In above both cases the query returns a non-authoritative reply because your ISP's DNS or Google's public DNS (8.8.8.8) are not authoritative for stackexchange.com
domain. As you have queried an non-authoritative nameserver the TTL value it provides will decrease for each time you query it. Once the TTL value expires the caching nameserver will requery the authoritative DNS server.
Authoritative query
So to get an authoritative reply you need to query the record from an authoritative DNS server and which can be found with below method.
$ dig ns stackexchange.com
...
;; ANSWER SECTION:
stackexchange.com. 84894 IN NS cf-dns02.stackexchange.com.
stackexchange.com. 84894 IN NS cf-dns01.stackexchange.com.
...
The ANSWER SECTION provides the authoritative nameservers for the domain stackexchange.com
and so if we need to get the authoritative reply then
$ dig stackexchange.com @cf-dns01.stackexchange.com.
While we query the authoritative DNS server the TTL values will not change because these Nameservers are the primary source of the information and they dont expire until its administrator changes it.
How ANY record works
ANY record is like a wild-card, you can use it to get all records that are cached/stored in a DNS server. For example I have queried stackexchange.com
for ANY record and my default DNS server replies as below.
$ dig any stackexchange.com
....
;; ANSWER SECTION:
stackexchange.com. 86350 IN SOA cf-dns01.stackexchange.com. dns.cloudflare.com. 2017456480 10000 2400 604800 3600
stackexchange.com. 176 IN A 198.252.206.140
stackexchange.com. 84338 IN NS cf-dns01.stackexchange.com.
stackexchange.com. 84338 IN NS cf-dns02.stackexchange.com.
....
Here you can see that the reply contains only information about SOA
, A
and NS
record. But there are actually more records for stackexchange.com
which are not cached in my default DNS server as I havent queried for it.
Now I am querying for MX
record to my default DNS server and the reply is as
$ dig MX stackexchange.com
....
;; ANSWER SECTION:
stackexchange.com. 300 IN MX 10 aspmx3.googlemail.com.
stackexchange.com. 300 IN MX 5 alt2.aspmx.l.google.com.
stackexchange.com. 300 IN MX 5 alt1.aspmx.l.google.com.
stackexchange.com. 300 IN MX 10 aspmx2.googlemail.com.
stackexchange.com. 300 IN MX 1 aspmx.l.google.com.
....
Now I again query for ANY
record and now you can see that query for ANY
has returned MX
records too. And so ANY
record will just provide records that are only cached on your default nameserver.
$ dig any stackexchange.com
....
;; ANSWER SECTION:
stackexchange.com. 298 IN MX 5 alt1.aspmx.l.google.com.
stackexchange.com. 298 IN MX 10 aspmx2.googlemail.com.
stackexchange.com. 86084 IN NS cf-dns01.stackexchange.com.
stackexchange.com. 298 IN MX 10 aspmx3.googlemail.com.
stackexchange.com. 298 IN MX 1 aspmx.l.google.com.
stackexchange.com. 298 IN MX 5 alt2.aspmx.l.google.com.
stackexchange.com. 86084 IN NS cf-dns02.stackexchange.com.
stackexchange.com. 243 IN A 198.252.206.140
stackexchange.com. 86343 IN SOA cf-dns01.stackexchange.com. dns.cloudflare.com. 2017456480 10000 2400 604800 3600
....
And as you can see the TTL values are changing for non-authoritative replies.
Best Answer
OK, I worked out what the problem is. When I run 'any' normally it takes its results from my local cache, so it will only display the records that I have already 'seen'.
To bypass this and retrieve all the records correctly I have to direct my query to the authoritative name server for that domain.
Here's the one-liner I'm now using to do it...
Gives (first time)...
Which is exactly the kind of output I need.
Enjoy!