Robots.txt Tester
Test your robots.txt file configurations. Verify whether specific pages are allowed or blocked for Googlebot, Bingbot, or custom AI crawlers.
Robots.txt Content
Test Parameters
How does matching work?
1. Specificity: The longest matching path pattern (based on string length) takes precedence over shorter rules.
2. Allow vs Disallow: If both Allow and Disallow rules match and have the same character length, the Allow rule takes precedence.
3. Wildcard: The `*` character represents any character sequence. The `$` represents the end of a URL path.