UltraScout AI › Learn › Can Claude access my website?

Can Claude access your website?

Anthropic's Claude uses ClaudeBot to crawl the web. To check if Claude can read your content:

Quick check with UltraScout

Use the free AI SEO Score tool — it automatically scans your robots.txt for:

• User-agent: ClaudeBot
• User-agent: anthropic-ai
• Any blocking directives

Manual check

Visit https://yourdomain.com/robots.txt and search for:

User-agent: ClaudeBot

• If followed by Disallow: / — Claude is blocked
• If followed by Allow: / — Claude is allowed
• If missing — Claude follows default rules (usually allowed)

How to allow ClaudeBot

Add to your robots.txt:

User-agent: ClaudeBot
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: Claude-Web
Allow: /

Why it matters

Claude is used by millions for research, content generation, and question answering. If Claude can't access your site, your expertise won't appear in Claude-powered answers.

How to block ClaudeBot (if needed)

User-agent: ClaudeBot
Disallow: /