Mercurial > defr > drupal > core
diff robots.txt @ 15:4347c45bb494 6.7
Drupal 6.7
author | Franck Deroche <webmaster@defr.org> |
---|---|
date | Tue, 23 Dec 2008 14:32:44 +0100 |
parents | c1f4ac30525a |
children |
line wrap: on
line diff
--- a/robots.txt Tue Dec 23 14:32:33 2008 +0100 +++ b/robots.txt Tue Dec 23 14:32:44 2008 +0100 @@ -1,4 +1,4 @@ -# $Id: robots.txt,v 1.9 2007/06/27 22:37:44 goba Exp $ +# $Id: robots.txt,v 1.9.2.1 2008/12/10 20:12:19 goba Exp $ # # robots.txt # @@ -20,27 +20,25 @@ User-agent: * Crawl-delay: 10 # Directories -Disallow: /database/ Disallow: /includes/ Disallow: /misc/ Disallow: /modules/ +Disallow: /profiles/ +Disallow: /scripts/ Disallow: /sites/ Disallow: /themes/ -Disallow: /scripts/ -Disallow: /updates/ -Disallow: /profiles/ # Files -Disallow: /xmlrpc.php +Disallow: /CHANGELOG.txt Disallow: /cron.php -Disallow: /update.php +Disallow: /INSTALL.mysql.txt +Disallow: /INSTALL.pgsql.txt Disallow: /install.php Disallow: /INSTALL.txt -Disallow: /INSTALL.mysql.txt -Disallow: /INSTALL.pgsql.txt -Disallow: /CHANGELOG.txt +Disallow: /LICENSE.txt Disallow: /MAINTAINERS.txt -Disallow: /LICENSE.txt +Disallow: /update.php Disallow: /UPGRADE.txt +Disallow: /xmlrpc.php # Paths (clean URLs) Disallow: /admin/ Disallow: /comment/reply/