# This is a standard robots.txt file that will allow ALL web spiders # to crawl your web site. # Visit http://www.searchengineworld.com/robots/robots_tutorial.htm # for more information on robot.txt files User-agent: * Disallow: