July Bot Rankings Flip; llms.txt Adoption Hits 51.8% With Zero Proven Lift
Cloudflare Radar data confirms ClaudeBot led every day of July 2026 while Anthropic's crawl-to-refer ratio improved sharply — yet a fresh panel study finds that publishing llms.txt produces no measurable citation benefit, forcing a hard reset on a widely adopted tactic.
This Week's Signals
ClaudeBot Leads GPTBot All 31 Days of July; 12-Month Rankings Fully Reversed
Why it matters for your score
ClaudeBot finished July at 16.28% of AI-bot traffic versus GPTBot's 9.74% — a complete reversal from July 2025, when GPTBot led 13.15% to 11.23%. If your site blocks Anthropic's crawler but allows OpenAI's, you are now restricting the higher-traffic bot. Review your robots.txt crawler permissions to reflect the current traffic hierarchy, not assumptions from a year ago.
Anthropic's Crawl-to-Refer Ratio Falls from 3,386:1 to 1,917:1 in One Month
Why it matters for your score
Anthropic's ratio is still roughly 400 times worse than Google's, meaning ClaudeBot crawls vastly more than it refers — but the direction of travel improved sharply in July. Separately, AI bot 403 enforcement hit 9.64% for the full month and 12.92% in the final week, so a growing share of sites are actively blocking crawlers. If your site is among them, you may be invisible to the bot currently winning the traffic race.
51.8% of Tracked Sites Publish llms.txt — Two Studies Find No Citation Lift
Why it matters for your score
A fixed panel of 219 hosts measured on August 3, 2026 found 113 publishing llms.txt — but two independent studies, one spanning roughly 300,000 domains and one reading server logs across roughly 900 sites, both show null citation lift. No verified fetch requests to llms.txt from any frontier AI lab were confirmed in the server-log study. Google has stated on the record in two separate forums that the file does nothing for Search, and as of mid-2026 the format has no W3C, IETF, or schema.org recognition.
What to do this week
- 1
Audit your robots.txt using Faro's /tools/robots-txt tool to confirm ClaudeBot is permitted — Cloudflare Radar data shows it now outpaces GPTBot every day of July 2026, and blocking it means missing the leading AI crawler.
- 2
Run your site through /tools/ai-readiness-scan to check whether your current crawler allowances match the July 2026 bot traffic hierarchy rather than year-old assumptions — Anthropic is now Cloudflare's #3 bot operator at 10.97% of all bot requests.
- 3
If you published llms.txt expecting citation gains, review what else on your site signals authority to AI systems — the August 2026 panel study finds no measurable citation lift from the file itself, so time spent on structured, data-rich content may produce better returns than maintaining the file.