How does Yahoo/Google find unknown domains?
Stephane Bortzmeyer
bortzmeyer at nic.fr
Tue Nov 9 03:21:25 UTC 2010
On Mon, Nov 08, 2010 at 02:02:52AM +0100,
Michelle Konzack <linux4michelle at tamay-dogan.net> wrote
a message of 94 lines which said:
> If I read the conditions of Networksolutions and Co, spidering of
> WHOIS records is prohibited also the commercial use of the data.
The list of domains in .COM (and other ICANN TLDs) can be obtained
legally:
http://www.verisign.com/domain-name-services/domain-information-center/tld-zone-access/index.html
> Now I see Googlebot, Yahoo and <he.net> quering my DNS Servers for
> exactly those domains.
Hurricane Electric uses these zone files for very interesting and
useful statistics:
http://bgp.he.net/ipv6-progress-report.cgi
More information about the bind-users
mailing list