approsto

Robots.txt generator

Online app for robots.txt generating.

By default all robots are

Crawl delay

Sitemap URL. Leave blank for none.

Robots are allowed to crawl your site:

Restricted directories or files. Relative to root, with trailing slash for dirs.

Your Robots.txt