24 Aug 20:15
Re: Search facility for the list archives?
> Date: Wed, 15 Aug 2007 14:04:58 -0700 > From: david.herron@... > ... > > It seems to me that going to google and entering: > > site:mail.openjdk.java.net search string > > would be a very good way to handle this. But there's a robot.txt saying > otherwise... > > wget http://mail.openjdk.java.net/robots.txt > > Yields: > > User-agent: * > Disallow: / > > There must be a reason for this to which I'm not privy. To me it makes > absolute perfect sense to index this stuff because it's the search > engines that draw the most traffic to a web site. You're completely right. There used to be a reason for that robots.txt. The new openjdk.java.net domains were actually online a few days before the launch at JavaOne, and we (obviously) didn't want search engines to index the site prior to the launch. We removed the robots.txt files from openjdk.java.net but forgot to removing the one on mail.o.j.n. I've now removed it. Give Google a couple of days, and you should be all set. - Mark
RSS Feed