Robots.txt Validator
A single malformed line can silently break crawling. This validator parses your robots.txt the way crawlers do, flags errors and warnings by severity with exact line numbers and fixes, and scores the file's overall health. Paste it or fetch it live.
What it checks
- Syntax errors — missing colons, malformed directives, invalid Sitemap URLs.
- Conflicts — a path that is both allowed and disallowed for the same agent.
- Best-practice warnings — missing sitemap, Crawl-delay: 0, duplicate rules.
- Risk flags — an accidental Disallow: / that would block your whole site.
Issues by severity
Not every issue is equal. The validator separates blocking errors from advisory warnings so you know what to fix now versus what's optional, and it gives every finding a line number and a recommended fix.
| Severity | Meaning | Example |
|---|---|---|
| Error | Breaks parsing or behavior | Disallow without a colon |
| Warning | Works, but risky or suboptimal | No Sitemap declared |
| Info | Worth knowing | AI crawler inherits the default |
How to validate your file
- Paste your robots.txt, or fetch it directly from your live domain.
- Review the issues grouped by severity, each with its line number.
- Apply the suggested fix, or rebuild cleanly in the Generator.
- Re-validate, then deploy with confidence.
Validation isn't testing
How do I check if my robots.txt is valid?
Paste your file into the validator or fetch it from your live site. It parses the file, flags syntax errors and best-practice issues by severity with line numbers, and gives an overall health score.
What does the validator catch?
Syntax errors (like missing colons or malformed sitemaps), conflicting allow/disallow rules, best-practice warnings (missing sitemap, Crawl-delay: 0, duplicates), and risky patterns such as an accidental Disallow: /.
Is the robots.txt validator free?
Yes. Validation is free and requires no sign-up. You can paste a file or fetch it live from any domain.
Does a valid robots.txt mean my URLs are crawlable?
Not necessarily. Validation confirms the file is well-formed and follows best practices, but a valid file can still block important URLs. Use the URL Tester to confirm specific pages are allowed.
Robots.txt Validator
Catch syntax errors and best-practice issues, with a health score.
Robots.txt Studio Editorial · Technical SEO & crawling
We build robots.txt tooling and parse thousands of real-world files. Guides are written by practitioners and reviewed against the Google and RFC 9309 specifications.