Contact
Contact the site owner for questions, corrections or source updates.
AdSense placeholder
For corrections, source updates or business questions, contact the site owner at the email configured in the admin settings.
FAQ
Does this guarantee that every crawler will obey?
No. robots.txt is a standard instruction for compliant crawlers, not a hard security barrier.
Can this hurt Google Search visibility?
Yes, if you block Googlebot or use broad rules carelessly. The checker highlights common SEO risks.
Where should robots.txt be uploaded?
It must be available at the root of the host, for example https://example.com/robots.txt.
Related tools
Related pages
AI Robots.txt Generator
Generate a clear robots.txt policy for AI crawlers without blocking regular search by accident.
Robots.txt Checker for AI CrawlersCheck whether AI crawlers, search bots and training bots can access your site.
Allow Search, Block AI Training robots.txt TemplateAllow search, block AI training can be controlled with robots.txt when the crawler respects the protocol. Use a specific rule, check the result, and avoid blocking Googlebot or Bingbot unless that is your real goal.
How to Block AI Crawlers with robots.txtBlock AI crawlers can be controlled with robots.txt when the crawler respects the protocol. Use a specific rule, check the result, and avoid blocking Googlebot or Bingbot unless that is your real goal.
Google-Extended vs GooglebotGoogle-Extended vs Googlebot can be controlled with robots.txt when the crawler respects the protocol. Use a specific rule, check the result, and avoid blocking Googlebot or Bingbot unless that is your real goal.