# # Robots.txt for SD4L drupal website # # More information about this file can be found at # http://www.robotstxt.org/wc/robots.html # A list of know bots can be found at # http://www.robotstxt.org/wc/active/html/index.html # For information about syntax checking see # http://www.sxw.org.uk/computing/robots/check.html # # 10-Oct-05 v0.1 RHB, original version # User-agent: * Crawl-Delay: 10 Disallow: /admin Disallow: /aggregator Disallow: /tracker Disallow: /comment/reply Disallow: /node/add Disallow: /user Disallow: /files Disallow: /search Disallow: /book/print Disallow: */add/