97% of llms.txt Files Ignored; AI Citations Vanish in 28 Days
Two new studies expose a gap between AI visibility tactics and actual AI crawler behavior: llms.txt files are almost universally skipped, and two-thirds of AI citations cycle out within a month. Businesses need to audit what AI crawlers actually read, not what the SEO industry assumed they would.
This Week's Signals
Ahrefs Study: AI Crawlers Skip llms.txt on 97% of the 137K Sites That Published One
Why it matters for your score
GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, and Google-Extended overwhelmingly bypass llms.txt and crawl HTML directly, meaning time spent creating and maintaining that file is largely wasted. Unlike robots.txt, llms.txt controls nothing and blocks nothing; the 'AI visibility' framing was attached by the SEO industry on speculation that AI platforms would reward the file. If you published an llms.txt expecting citation lift, shift your effort toward the HTML signals crawlers are actually reading.
What to do this week
- 1
Run your site through Faro's AI Readiness Scan (/tools/ai-readiness-scan) to confirm what AI crawlers actually fetch from your pages, the Ahrefs study shows they read HTML directly, so your on-page structure matters far more than any llms.txt file.
- 2
Audit and expand your schema markup using Faro's AI Schema tool (/tools/ai-schema), since schema is identified as the highest-impact technical signal for Google AI Overview citation eligibility and the organic-to-AIO citation gap has widened significantly.
- 3
Use Faro's Competitor Intelligence tool (/tools/competitor-intelligence) to track which of your competitors hold AI citations week over week; with a 66% citation replacement rate over 28 days, monitoring citation stability is now an ongoing task, not a one-time check.