[svnbook commit] r2798 - trunk/www

[cmpilato]

Author: cmpilato
Date: Fri May 11 01:44:39 2007
New Revision: 2798

Log:
Allow Google's Adsense crawlers to crawl the nightly area (while still
disallowing general indexing of that area.

Modified:
   trunk/www/robots.txt

Modified: trunk/www/robots.txt
==============================================================================
--- trunk/www/robots.txt	(original)
+++ trunk/www/robots.txt	Fri May 11 01:44:39 2007
@@ -3,3 +3,6 @@
 Disallow: /viewcvs
 Disallow: /cgi-bin
 Disallow: /trac
+
+User-Agent: MediaPartners-Google
+Allow: /nightly
\ No newline at end of file