Auth0

No Content-Signal header

No Content-Signal header. AI systems have to guess whether they're allowed to use your content for training, search, or as agent input. Declaring your policy proactively builds trust with AI platforms.

Fix: Add a Content-Signal HTTP response header to tell AI systems your content permissions: Content-Signal: ai-train=yes, search=yes, ai-input=yes In Next.js (next.config.js): headers: async () => [{ source: '/(.*)', headers: [{ key: 'Content-Signal', value: 'ai-train=yes, search=yes, ai-input=yes' }] }] This header is part of the emerging Content Signals standard (contentsignals.org) adopted by Cloudflare and others. It declares explicitly what AI can do with your content.

No Markdown content negotiation

Server doesn't return Markdown when requested with Accept: text/markdown. AI agents consume HTML which wastes context and introduces parsing errors. Markdown responses are the emerging standard for agent-optimised content delivery.

Fix: Enable Markdown content negotiation so AI agents can request clean, token-efficient versions of your pages. Quickest path: enable Cloudflare's 'Markdown for Agents' feature (Pro plan+) which auto-converts HTML to Markdown at the CDN edge when an AI agent requests it. No origin changes required. Alternatively, in Next.js you can detect the Accept header and serve .md files: if (req.headers.accept?.includes('text/markdown')) { res.setHeader('Content-Type', 'text/markdown'); } This is one of the highest-leverage changes you can make for AI agent accessibility.

No HTTP Link discovery header

No HTTP Link header advertising discovery files. AI agents that don't know to look for llms.txt won't find it. A Link header in the HTTP response signals discovery files proactively , before the agent even reads the page.

Fix: Add a Link header to your HTTP responses pointing to your discovery files: Link: </llms.txt>; rel="ai-content", </sitemap.xml>; rel="sitemap" In Next.js (next.config.js): headers: async () => [{ source: '/(.*)', headers: [{ key: 'Link', value: '</llms.txt>; rel="ai-content", </sitemap.xml>; rel="sitemap"' }] }] This is the HTTP equivalent of a signpost , an agent making its first request sees where to find your llms.txt immediately.

OKF knowledge bundle (/okf/index.md)

No OKF bundle. Google's Open Knowledge Format (June 2026) lets AI agents navigate your business knowledge as a graph of linked markdown files. Less than 0.1% of sites have this.

Fix: Create an OKF bundle starting with /okf/index.md. Use Faro's OKF Generator to build a complete bundle automatically.

/agents.json (agent capability manifest)

No /agents.json. This file tells AI agents what actions they can take, what APIs you expose, and how to interact with your business.

Fix: Create /agents.json at your root: { "name": "Your Company", "description": "What you do", "url": "https://yourdomain.com", "capabilities": ["Browse products", "Check pricing", "Start trial"], "api": { "docs": "https://yourdomain.com/docs" }, "pricing_url": "https://yourdomain.com/pricing", "auth": ["api_key"] }