By Nikhil Kumar, founder of MentionsAPI. Last updated July 8, 2026.
Everyone is adding llms.txt. Almost nobody agrees on whether it does anything.
So here is the honest version. llms.txt is a proposed standard: a Markdown file at yourdomain.com/llms.txt that tells AI models which of your pages matter and summarizes what each one covers. Jeremy Howard proposed it in September 2024, per Search Engine Land. It will not lift your rankings, and Google has said it will not use it. But it costs almost nothing, and the tools that do read it are growing.
What is llms.txt?
llms.txt is a single Markdown file that points AI models to the content on your site worth reading. Instead of leaving a model to crawl everything and guess, you hand it a curated map: a summary of who you are and a short list of your key pages. Think of it as a business-to-agent file, the machine-readable version of a site menu built for AI.
It lives at the root of your domain, the same place as robots.txt. That is the whole idea: one predictable location an AI can check to understand your site fast.
What does an llms.txt file look like?
An llms.txt file is plain Markdown with a set structure: an H1 with your site name, a one-line summary in a blockquote, then sections of links, each with a short note on what it covers. An optional llms-full.txt holds your full content inlined for models that want everything.
Here is a minimal example:
# MentionsAPI
> One API to check if AI engines mention and cite your brand.
## Docs
- [Quickstart](https://mentionsapi.com/docs): set up in 5 minutes
- [API reference](https://mentionsapi.com/docs/api): every endpoint
## Guides
- [What is AI visibility](https://mentionsapi.com/blog/what-is-ai-visibility): measure it across LLMsllms.txt vs robots.txt: what is the difference?
They do different jobs. robots.txt controls access, telling crawlers what they may fetch, and every major AI crawler respects it. llms.txt controls comprehension, suggesting which content is worth reading. One gates, the other guides.
| robots.txt | llms.txt | |
|---|---|---|
| Job | Control access | Guide comprehension |
| Respected by | All major AI crawlers | Dev tools and agents |
| Enforced? | Yes, universally | No, proposed only |
| Use it to | Allow or block crawlers | Point AI at your best pages |
How do you create an llms.txt file?
Creating one takes about fifteen minutes. List your most important pages, write the Markdown, place the file at your domain root, and validate it. A generator can draft it, but hand-pick the links, since a curated file beats a dump of every URL.
Does llms.txt actually work in 2026?
Honestly, mostly not yet, and it depends what you mean by work. About 10% of sites have adopted it, per a SE Ranking study of 300,000 domains, and adopters include Anthropic, Stripe, Cloudflare, and Vercel. But answer engines barely touch it: Otterly found only about 0.1% of AI bot requests target llms.txt.
Where it clearly does work is developer tooling. AI coding assistants like Cursor, GitHub Copilot, and Claude Code, plus MCP agents, fetch llms.txt to read your docs with less wasted context. If you publish documentation, that alone can justify it.
llms.txt is not a ranking hack. It is a cheap, sensible bet that the standard sticks, and free help for the AI tools that already read it.The honest verdict
Should you add an llms.txt file?
For most sites, yes, but as insurance, not a growth lever. It costs about fifteen minutes and carries no risk, and it genuinely helps if you have docs that agents and IDE tools read. What it will not do is lift your Google rankings or make ChatGPT cite you.
One priority note: if you care about AI crawlers, sort out robots.txt before llms.txt. That is the file every AI bot actually obeys, and it decides whether they can reach you at all. The real work still lives in our AI search optimization guide.
Frequently asked questions
What is llms.txt used for?
Is llms.txt the same as robots.txt?
Do ChatGPT and Perplexity read llms.txt?
Does Google support llms.txt?
How do I create an llms.txt file?
Is llms.txt worth it in 2026?
Add it, then measure what matters
llms.txt is a fifteen-minute file that might matter more later than it does today. Add it for the dev tools that read it now, keep your robots.txt clean, and do not expect it to move rankings.
Then measure the thing that actually counts: whether AI engines cite you. Pull that baseline with MentionsAPI and track it as the standard, and your visibility, evolve.