Let's talk(847) 414-4272

Glossary / SEO & AI Search

AI crawler.

A bot that reads websites to train or power AI systems and answers.

What it means

A bot that reads websites to train or power AI systems and answers. Examples include GPTBot, ClaudeBot and PerplexityBot. Whether to allow them is controlled in robots.txt; blocking them can keep your content out of AI answers.

Related terms

More in SEO & AI Search

Questions aboutai crawler?

Ask the team that does this work every day. We reply within one business day.