Free tool · runs in your browser
AI robots.txt generator
Choose which of 18 AI crawlers may read your site. Keep answer engines in, opt out of training if you want — and get a file that’s correct.
Your file
Nothing you type leaves your browser.
To be cited by AI assistants, robots.txt must allow their search and user-request crawlers. This generator keeps all of them allowed under User-agent: * and adds a separate Disallow group only for the bots you untick, plus your sitemap location.
Before you publish
- Unticking a search or user-requested bot removes you from that assistant’s answers.
- Check your firewall or CDN doesn’t block bots that robots.txt allows.
See the AI crawler directory for what each bot does.
Free AI visibility audit
Check it’s working
Run a free audit after you publish — we test the live file.
No sign-upNo cardReads up to 8 public pages