# robots.txt, www.co2alarm.com 05/09/2009 # User-agent: * Disallow: /clusters/ Disallow: /clusterings/search_term/ Disallow: /clusterings/search/ Crawl-delay: 18 User-agent: Slurp Disallow: / User-agent: WebReaper Disallow: / Sitemap: http://www.co2alarm.com/sitemap.xml