robots.txt Generator

Name: robots.txt Generator
Rating: 4.8 (50 reviews)
Author: Marco Orta

Build your site's robots.txt with per-robot rules (Allow, Disallow, Crawl-delay) and sitemaps, with quick templates. Copy or download instantly.

Templates

User-agent

Disallow · do not crawl · one path per line

Allow · exceptions · optional

Crawl-delay (sec) · optional

Sitemaps · one URL per line

robots.txt

User-agent: *
Disallow:

Sitemap: https://yourdomain.com/sitemap.xml

Everything is generated in your browser; nothing is sent to any server.