aboutsummaryrefslogtreecommitdiff
path: root/en_US.ISO8859-1/htdocs/robots.txt
Commit message (Expand)AuthorAgeFilesLines
* remove testing robots.txt entry: it didn't help :(Eitan Adler2015-04-031-1/+0
* Re-add /cgi/url.cgi to the disallowed list, this is now served by Varnish.Gavin Atkinson2014-07-231-0/+1
* As of r45331, /cgi/pds.cgi and /cgi/url.cgi no longer exist.Gavin Atkinson2014-07-221-2/+0
* Re-enable indexing manual pages, jet is fixed.Glen Barber2013-12-291-1/+1
* Block man.cgi lookups until jet stops overheating.Glen Barber2013-12-291-1/+1
* Remove the restriction of indexing man.cgi.Glen Barber2013-10-041-1/+1
* Explicitly list the files within cgi/, since not all crawlers willGlen Barber2013-10-041-1/+16
* Remove a stray blank line from robots.txt.Glen Barber2013-10-041-1/+1
* Due to the time sensitive nature of some of the testing I'm doing, readd /sit...Eitan Adler2013-06-011-0/+1
* Sigh...Glen Barber2013-06-011-1/+0
* The folder blocked previously seems to be a special case of a broader problem...Eitan Adler2013-05-311-2/+1
* There appear to be configuration errors causing an absurd amount of links to ...Eitan Adler2013-05-311-0/+4
* - Remove junk directories.Hiroki Sato2012-05-171-0/+4