Building Web Site Directory

[john at thinman.com]

Hello all,

I am in the process of using a lot to tools to create a open directory
of all the worlds web sites, what they do and what people think about
them.

Garnering info about the web sites and comments people may have made
about them is actually easier than it sounds using Perl in an AI sort of
way.  Much of the information is available via whois.

But... getting the list of the www servers, ie their domain names has
proved harder, for the obvious reason that it is the first step the
kiddie attackers take to develop hit lists.

I would be very grateful for any ideas on how to get lists of the domain
name resolutions w/o doing xfers.

We actually probed a couple of name servers yesterday basically to
figure out what is going on.  Today we found out that security folks
recommend logging unauthorized xfers, oops.

Thanks, anyone !



Sent via Deja.com http://www.deja.com/
Before you buy.