robots.txt

Path: examples/rails_mysql_example/public/robots.txt
Last Update: Fri May 23 01:37:10 -0700 2008

See www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file

To ban all spiders from the entire site uncomment the next two lines: User-Agent: * Disallow: /

[Validate]