Italiano
Check syntax, user-agent groups, allow and disallow rules, declared sitemaps and URL access for a specific crawler.
Enter a URL or paste the content. When both fields are filled, the pasted content is analyzed.
Create a clean starting point to review before publishing. The generator does not modify your website.
A robots.txt checker helps interpret the instructions a website sends to crawlers. The file does not directly decide which pages can be indexed: it primarily controls crawling. Review it together with meta robots, canonical tags, HTTP status and XML sitemaps.
The same path can be allowed for Googlebot and blocked for another user-agent. The tester selects the most specific group, applies wildcards and end anchors, and evaluates the longest matching allow or disallow rule.
Groups without a user-agent, malformed directives, overly broad paths and accidental blocks on CSS, JavaScript or strategic sections can restrict crawling. A Disallow rule is not a reliable way to remove a URL from Google.
Declaring a sitemap in robots.txt helps crawlers discover the XML file. Continue with the sitemap checker and canonical/indexability checker to verify that crawling and indexing signals agree.
A robots.txt generator creates valid syntax but cannot know the SEO value of your directories. Before publishing, review templates, private areas, parameters, rendering resources and differences between staging and production.
Enter the domain or robots.txt URL, choose a user-agent and optionally add a URL from the same site. The tester reads the groups and shows which allow or disallow rule determines the result.
Not reliably. Robots.txt restricts crawling, but a blocked URL can still be discovered and indexed through external links. Use an accessible noindex directive or another appropriate removal method when indexing must be prevented.
A Sitemap directive can sit outside user-agent groups and must contain an absolute URL, for example Sitemap: https://www.example.com/sitemap.xml.
