Anthropic MCP Goes GA as llms.txt Adoption Crosses 50K Domains
Anthropic's Model Context Protocol reached general availability this month with expanded enterprise tooling, while llms.txt indexing hit a meaningful adoption threshold — signaling that AI-agent-readable infrastructure is shifting from early-adopter experiment to baseline expectation.
This Week's Signals
Anthropic MCP 1.0 GA Ships with OAuth 2.1 and Enterprise Registry Support
Why it matters for your score
MCP 1.0's GA release includes OAuth 2.1 authentication and a forthcoming public MCP server registry — meaning Claude agents can now discover and authenticate against business-exposed APIs in a standardized way. If your business has a service API and it's not MCP-registered, Claude agents will route around you to competitors who are. This is the moment to treat MCP exposure as a distribution channel, not a developer curiosity.
Perplexity Confirms Citation Weighting Favors Structured JSON-LD and llms.txt Presence
Why it matters for your score
Perplexity's engineering team confirmed in a June post that their citation ranker applies a measurable lift to pages with valid JSON-LD schema and domains with an llms.txt file — treating both as trust signals that reduce hallucination risk. For business owners, this is the clearest public acknowledgment yet that AI-structured markup directly influences whether you get cited or ignored. Pages without schema are now structurally disadvantaged in Perplexity answers, not just theoretically.
GPTBot Crawl Frequency Now Tied to robots.txt AI Directives, OpenAI Confirms
Why it matters for your score
OpenAI updated its crawler documentation confirming that GPTBot now adjusts recrawl priority based on explicit AI-permissive signals in robots.txt — sites that actively allow GPTBot with crawl-delay hints are recrawled more frequently and have their updates reflected in ChatGPT knowledge faster. Passive silence in robots.txt no longer equals equal treatment; you are either signaling readiness or being deprioritized. This makes robots.txt configuration an active freshness lever, not a passive block list.
What to do this week
- 1
Register your business API as an MCP server this week — even a minimal endpoint exposing your services, pricing, and availability puts you in the pool Claude agents query. Use Faro's /tools/ai-readiness-scan to identify whether your current API surface is MCP-compatible and what gaps to close first.
- 2
Audit and update your robots.txt to include explicit GPTBot and PerplexityBot allow directives with crawl-delay values — this is now confirmed to affect both recrawl frequency and citation weighting. Faro's /tools/robots-txt generator outputs the correct syntax for all major AI crawlers including ClaudeBot, GPTBot, and PerplexityBot in one pass.
- 3
Add or validate your llms.txt file and JSON-LD schema before end of week given Perplexity's confirmed citation weighting — a missing or malformed llms.txt is now a measurable citation penalty, not a theoretical one. Use Faro's /tools/llms-txt and /tools/ai-schema tools to generate both, then re-run /tools/ai-readiness-scan to confirm your domain registers the trust signals Perplexity's ranker is looking for.