Question about visibility

[Hardy, Andrew]

Further to the original post, as well as not creating a DNS record and
"possibly" adding robot.txt with appropriate content, as discussed, I
presume that if I run the http server on a personally selected unprivileged
port then it is very "unlikely" the site pages will be
indexed/discovered/etc surely?

Thoughts?

Thanks.


On Sun, Oct 21, 2018, 20:32 N6ghost <n6ghost at gmail.com> wrote:

> On Thu, 11 Oct 2018 15:39:55 -0400
> Barry Margolin <barmar at alum.mit.edu> wrote:
>
> > In article <mailman.671.1539286015.803.bind-users at lists.isc.org>,
> >  Dennis Clarke <dclarke at blastwave.org> wrote:
> >
> > > On 10/11/2018 03:21 PM, Leonardo Rodrigues wrote:
> > > > Em 11/10/18 16:13, Barry Margolin escreveu:
> > > >>
> > > >> If you accidentally, or someone else intentionally, create a
> > > >> link to the site that uses the IP and put it on a web page that
> > > >> Google can get to, it will probably find the page.
> > > >>
> > > >>
> > > >
> > > >      robots.txt, on your website root, is your friend. Simply
> > > > deny web crawling on it, and you're (probably) done.
> > > >
> > >
> > > If you believe robots.txt means anything at all.
> >
> > Google is known to obey it, and the question was about avoiding
> > getting your site indexed by Google.
> >
> > Of course, that doesn't mean someone won't find the site on their
> > own. If the link to it is on some other page that isn't blocked by
> > robots.txt, someone might stuble across that page and then click on
> > the link.
> >
> > But if you're mainly worried about someone googling the words that
> > are on your website and Google sending them to the development
> > version instead of the production version, you're pretty safe.
> >
> > Actually, DNS has very little impact on this at all. AFAIK, Google
> > doesn't crawl DNS, it just crawls web pages and follows links. My
> > company's development server is in DNS, and it's not firewalled (we
> > all work from our homes, there's no company network to restrict
> > access with), but I've never heard of anyone accidentally being
> > directed there by Google, because we don't publish links to this
> > server.
> >
>
> robot.txt is suppose to govern whats indexed... not sure how well its
> followed nowadays but thats the process for it.
> _______________________________________________
> Please visit https://lists.isc.org/mailman/listinfo/bind-users to
> unsubscribe from this list
>
> bind-users mailing list
> bind-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20181024/6f18e5a2/attachment.html>