[svnbook] r3819 committed - Tell web crawlers to stop crawling the older versions of the book.

Daniel Shahaf danielsh at apache.org
Wed Jan 5 08:25:08 CST 2011


+1!

svnbook at googlecode.com wrote on Wed, Jan 05, 2011 at 13:55:25 +0000:
> Revision: 3819
> Author: cmpilato at gmail.com
> Date: Wed Jan  5 05:54:22 2011
> Log: Tell web crawlers to stop crawling the older versions of the book.
> http://code.google.com/p/svnbook/source/detail?r=3819
>
> Modified:
>  /trunk/www/robots.txt
>
> =======================================
> --- /trunk/www/robots.txt	Thu May 10 23:44:39 2007
> +++ /trunk/www/robots.txt	Wed Jan  5 05:54:22 2011
> @@ -3,6 +3,10 @@
>  Disallow: /viewcvs
>  Disallow: /cgi-bin
>  Disallow: /trac
> +Disallow: /en/1.0
> +Disallow: /en/1.1
> +Disallow: /en/1.2
> +Disallow: /en/1.4
>
>  User-Agent: MediaPartners-Google
>  Allow: /nightly
>
> _______________________________________________
> svnbook-dev mailing list
> svnbook-dev at red-bean.com
> http://www.red-bean.com/mailman/listinfo/svnbook-dev




More information about the svnbook-dev mailing list