DeveloperSearch & find

Get AI Crawler Robots Rules on Automatelab

Return robots.txt rules that allow the major AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended) to access a site.

Automatelab source site10 tools on this site

What this tool does

Return robots.txt rules that allow the major AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended) to access a site. Use it on Automatelab with an agent. It runs in the browser on the website and needs no package install.

How to install

Open the automatelab.tech website and ask your browser agent to use get_ai_crawler_robots_rules

Run with an agent

Use the get_ai_crawler_robots_rules tool on automatelab.tech. Return the structured result and cite the source page.

Input contract

Verify the live listing at the source before a production call. Fields marked as a starter contract may differ from what the site expects.

Open the source definition