Why isn't a new DNS record immediately available globally?

Question

Simple case:

I register a domain
Specify 2 name servers
Setup web hosting
Visit the domain in my browser

At this point the browser tells me it cannot resolve the host, presumably because the DNS record has not propagated yet (i.e. cleared collective DNS server caches)...

This is strange to me though.

Why isn't the domain immediately resolved in the browser? Why can't the browser/router/name-servers simply forward the DNS query to the authoritative server instead of returning the cached query results which seems to be "Could not find host, yourNewDomain.tld"

I am probably greatly misunderstanding something here. I'd really appreciate some clarity.

Thanks

You have a number of good answers available to you right now. Before analyzing any of them too deeply, I recommend running dig +trace +additional yourdomain.com (or a web equivilant) to narrow down which of these problems are in play. This will also help you spot any mistakes you might have made with your registrar settings. — Andrew B, Apr 06 '13 at 16:40

score 2 · Answer 1 · answered Apr 06 '13 at 15:09

2

That's what the TTL is for. It specifies the amount of time before a nameserver will clear its cache and fetch new information.

answered Apr 06 '13 at 15:09

tdk2fe

600

Is a TTL really that useful for domains that the name-server doesn't know exists? If a DNS query comes in, and the name-server does not have it available at all, would it not be possible to escalate the query up to other name servers immediately? – JIP Apr 06 '13 at 15:15
Take the .com TLD for example. The main nameserver for this has a TTL of 900 which means that a client or recursing nameserver will ask again only after 15 minutes. This is a good thing to keep the load down from repeated requests for nonexisting domains (e.g. due to a typo). – Sven Apr 06 '13 at 15:19
@SvW Ah ok, that makes sense I guess. So caching & TTL is not just for efficiency but to also prevent repeatedly hammering main nameservers? – JIP Apr 06 '13 at 15:22
@JIP: Among other things, yes. – Sven Apr 06 '13 at 15:25
TTL does not apply for records which do not exist. – Andrew B Apr 06 '13 at 16:34
DNS servers may cache non-existent query results, that depends on the configuration of the TLD's name servers. I would assume that most servers do not cache those results - or only for a short time (seconds to minutes). – Lukas Apr 06 '13 at 16:39
@Lukas Correct, but that's negative caching, not TTL. – Andrew B Apr 06 '13 at 16:45
@AndrewB you are correct. However, he asked why it isn't immediately available after he registers the domain. – tdk2fe Apr 06 '13 at 20:04
@tdk2fe TTL is how long you wait when you've successfully fetched the information in question. If the record is not in cache and has never been requested before, it will result in an authoritative lookup (barring a lame NS). At this point it's either there to cache (with TTL) or it gets negative cached (based on SOA). – Andrew B Apr 06 '13 at 20:10

score 2 · Answer 2 · answered Apr 06 '13 at 16:05

Q: Why can't the browser/router/name-servers simply forward the DNS query to the authoritative server instead of returning the cached query results which seems to be "Could not find host, yourNewDomain.tld"

A: Because your name servers haven't been "published" from your Registrar to the parent server(s). Assuming your domain is a .com, your registrar needs to "publish" the name servers for your domain to the .com gTLD servers, meaning the gTLD servers don't know what name servers are authoritative for your domain until your Registrar "publishes" that information to the .com gTLD servers.

In addition, the name servers aren't returning cached results, they're returning no results, or in other words they're returning NXDOMAIN (non-existent domain) because for all intents and purposes your domain doesn't exist until the parent servers know about your name servers.

The TLD providers may also delay record publication on their servers. As an example, DENIC refreshes its DNS servers roughly every two hours (I believe this was six hours in the past). Until this is done, the domain will not "exist". — Lukas, Apr 06 '13 at 16:44

score 2 · Answer 3 · edited Apr 13 '17 at 12:14

Your problem is going to be one of two things:

Domain hasn't been published from your registrar yet:
@joeqwerty and @YLearn have covered this pretty well.
Domain has been published, but you or someone else asked for a record before it was on the authoritative nameserver:
This is known as negative caching and is explained below.

There are two common caching concepts in DNS:

Caching:
This is what most people are familiar with. When a caching server obtains an answer from an authoritative nameserver, the caching server keeps the response in memory for the duration of the TTL. The caching server will report a TTL for the record of (total TTL - how long record has been in cache). You can determine the original TTL by asking the authoritative nameserver for the same record.
Negative caching:
The concept people are less familiar with. This is like caching, but designed to prevent caching servers from overworking themselves when records don't exist. Since there is no record, there is no TTL: instead the duration of how long a record should be negatively cached is determined from the last numeric field of the SOA record.

-

$ dig +noall +answer serverfault.com SOA
serverfault.com.        3600    IN      SOA     ns1.serverfault.com. sysadmins.stackoverflow.com. 2013020902 600 600 604800 1440

In the above example, the negative caching interval for serverfault.com is 1440 seconds. If you ask a caching server for the record before you've published it, you risk it getting negative cached.

Related answer: How long does negative caching typically last?

score 1 · Answer 4 · answered Apr 06 '13 at 16:04

Your domain needs to be inserted into the top level DNS servers. When you make a DNS request, for example for www.thisismyexampledomain.com your DNS client goes to your recursive DNS and asks for the information.

Your DNS then has to go looking for the answer and here is the simplified process. It starts at the very top, the assumed "." at the end of your request. This will direct your DNS to the servers responsible for "com". When it checks with those, they will direct your DNS to the servers responsible for "thisismyexampledomain". Your DNS will then get the answer for the hostname of "www" from those servers and return it to your computer.

The top level domain servers are managed by a number of different companies around the world, and as you can probably guess can be very busy doing what they do and are asked to make a large number of changes each day. These companies batch the changes to go in at certain times for a number of reasons.

Why isn't a new DNS record immediately available globally?

4 Answers4