If you’ve spent any time on AI SEO Twitter or LinkedIn in the last twelve months, you’ve seen the same pitch. Drop an llms.txt at your domain root, ship a tidy Markdown index of your “AI-friendly” content, and watch the citations roll in. It’s positioned as the robots.txt of the LLM era — a five-minute file that supposedly tells ChatGPT, Perplexity, and Claude what on your site is worth ingesting.
It does not.
After thirty years of watching standards get declared, adopted, and quietly buried, this one is — at least for now — closer to fan fiction than infrastructure. Server-log audits across 2025 and into 2026 keep arriving at the same conclusion: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and the rest of the major retrieval crawlers do not request `/llms.txt`. John Mueller has said so publicly. No major LLM provider has officially adopted it. The aeo.press 2026 audit and the aeoengine.ai crawl-log study both put real-world fetches at effectively zero. The file is a proposal someone hoped would become a spec. It hasn’t.
The mechanic — what AI crawlers actually look at
The retrieval pipelines behind ChatGPT, Perplexity, Gemini, and Google AI Overviews are not opening a curated text file you wrote for them. They are doing one of two things, depending on the engine.
First, they crawl your HTML the way classic search has for two decades. They follow links, parse the DOM, extract text, build embeddings. Whether you have clean `
` markup, semantic HTML, schema, and an actual content hierarchy matters far more than whether you’ve published a manifest. A page with strict H1 → H2 → H3 nesting and a 40–60 word answer in the first paragraph does the work llms.txt was supposed to do — and the bots are already reading it.Second, they retrieve from third-party indexes you don’t control. ChatGPT pulls heavily from Wikipedia (~48% of cited pages) and Reddit. Google AI Overviews leans on Reddit, YouTube, and Quora. Perplexity is even more Reddit-dependent at ~47%. Your llms.txt has zero influence on any of this. What does have influence: your brand presence in those properties, your entity consistency on Wikidata, and whether your owned content can be parsed cleanly when the bot eventually arrives via a regular HTTP request to a real URL.
There is a softer counterargument worth acknowledging. Maybe llms.txt is symbolic — a signal of intent, the way `humans.txt` was. Maybe one of the majors adopts it next year. Maybe. Treat it like meta keywords in 2002: costs nothing to ship, but don’t budget time around it, don’t charge clients for it, and don’t let it sit on your roadmap in place of work that demonstrably moves the needle.
What to do this week instead
Spend the half hour you would have spent debating llms.txt on four things that actually correlate with citations.
1. Audit your H1/H2/H3 nesting. 68.7% of pages cited by AI engines follow strict heading hierarchy. Crawl your top 20 commercial pages and find every place an `
` sits under an `
` with no `
` between them. Fix the nesting first — you’ll often discover the answerable units the bots want were always there, just buried under sloppy markup.
` between them. Fix the nesting first — you’ll often discover the answerable units the bots want were always there, just buried under sloppy markup.
2. Front-load a 40–60 word answer. 44.2% of LLM citations come from the first 30% of a page. If your most cite-worthy claim is in paragraph six, move it to paragraph one. Let the rest of the page do the supporting work.
3. Ship Article, FAQ, and Organization schema. Schema is not a silver bullet — and don’t let anyone tell you it is — but Google and Microsoft both confirmed in 2025 that structured data helps AI understanding. It pairs with the heading work above and costs less than a day of dev time on most stacks.
4. Tighten your entity hygiene. Same bio, same NAP, same author markup across your site, LinkedIn, Wikidata, and Crunchbase. Entity authority is replacing keyword authority. Inconsistent entity data is a slower-bleeding but much bigger leak than any missing manifest file.
Paris Roussos has been doing SEO since 1996 (co-founded a Forbes Best of the Web–winning site back in the day) and now runs a white-label AI SEO practice for agencies and brands — flat-rate, $500–$1,500/mo per client. If your top-of-funnel traffic is leaking into ChatGPT and Perplexity and you want it back, email parisroussos@gmail.com.
If a tactic is being sold as the new robots.txt and the major bots aren’t even fetching it, that’s not optimization — that’s theater.