Figma

GPTBot (ChatGPT) is blocked

GPTBot is blocked. ChatGPT cannot learn from your site , competitors who allow it get recommended instead.

Fix: In your robots.txt, add: User-agent: GPTBot Allow: / Place above any catch-all Disallow rules.

ClaudeBot (Anthropic) is blocked

ClaudeBot appears blocked. Claude won't have fresh data about your products.

Fix: Add both Anthropic crawlers to your robots.txt: User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: /

PerplexityBot is blocked

PerplexityBot appears blocked. You won't show up when people use Perplexity to research your category.

Fix: Add to your robots.txt: User-agent: PerplexityBot Allow: /

Google-Extended (Gemini) is blocked

Google-Extended appears blocked. Your content won't appear in Google's Gemini responses or AI Overviews , a large and growing traffic source.

Fix: Add to your robots.txt: User-agent: Google-Extended Allow: / Google-Extended is the crawl identity used for Gemini, AI Overviews, and other Google AI products. Blocking it removes you from Google's AI answers.

No Content-Signal header

No Content-Signal header. AI systems have to guess whether they're allowed to use your content for training, search, or as agent input. Declaring your policy proactively builds trust with AI platforms.

Fix: Add a Content-Signal HTTP response header to tell AI systems your content permissions: Content-Signal: ai-train=yes, search=yes, ai-input=yes In Next.js (next.config.js): headers: async () => [{ source: '/(.*)', headers: [{ key: 'Content-Signal', value: 'ai-train=yes, search=yes, ai-input=yes' }] }] This header is part of the emerging Content Signals standard (contentsignals.org) adopted by Cloudflare and others. It declares explicitly what AI can do with your content.