[svnbook] r4120 committed - Disallow crawling of /en/1.5, and remove some old noop rules.
svnbook at googlecode.com
svnbook at googlecode.com
Fri Oct 21 13:24:40 CDT 2011
Revision: 4120
Author: cmpilato at gmail.com
Date: Fri Oct 21 11:23:56 2011
Log: Disallow crawling of /en/1.5, and remove some old noop rules.
http://code.google.com/p/svnbook/source/detail?r=4120
Modified:
/www/robots.txt
=======================================
--- /www/robots.txt Wed Jan 5 05:54:22 2011
+++ /www/robots.txt Fri Oct 21 11:23:56 2011
@@ -1,12 +1,11 @@
User-agent: *
Disallow: /nightly
-Disallow: /viewcvs
-Disallow: /cgi-bin
Disallow: /trac
Disallow: /en/1.0
Disallow: /en/1.1
Disallow: /en/1.2
Disallow: /en/1.4
+Disallow: /en/1.5
User-Agent: MediaPartners-Google
Allow: /nightly
More information about the svnbook-dev
mailing list