What Is llms.txt? A Plain-English Guide
llms.txt is a plain text file, similar to the robots.txt file most websites already have, that gives AI systems like ChatGPT and Claude a short, organized map of a website’s most useful pages. It sits at yourdomain.com/llms.txt and lists your key pages in plain language, so an AI assistant researching your business doesn’t have to guess which parts of your site actually matter.
It’s a new idea, and it isn’t a Google ranking factor. It’s aimed at AI systems that read websites to answer questions, not traditional search crawlers. Here is what it looks like, who reads it, and whether it’s worth your time.
Key takeaways
- llms.txt is a plain text file at yourdomain.com/llms.txt that lists your best pages for AI systems. It’s a 2024 proposal, not an official standard.
- Google says you don’t need AI text files to appear in its AI features, and no one publishes which AI systems read it.
- Treat it as a low-cost bonus after the basics: your Google profile, clear pages, matching details and reviews.
- It doesn’t block or allow anything. Use robots.txt to control crawlers.
Where it came from
The proposal comes from Jeremy Howard, and the llmstxt.org page records the original as published on September 3, 2024. It’s a proposal, not an official standard from Google, OpenAI or any standards body.
What it actually looks like
An llms.txt file is a short Markdown document. According to the llmstxt.org format, it has a title as a top-level heading (the only required part), an optional one-sentence summary, and then lists of links, each with a short note. A local bakery’s might look like this:
# Forge & Vine Bakery
> A from-scratch bakery in Groton, MA, serving fresh bread, pastries, and custom cakes.
## Pages
- [Menu](https://example.com/menu): Full bakery menu, updated daily
- [Order a cake](https://example.com/cakes): Custom cake ordering, 1 week notice required
- [Hours & location](https://example.com/visit): Store hours, address, and parking
That’s it. No code and no special software. By convention, a section called “Optional” can hold links an AI can skip when it needs less detail. You can see a real one on our own site at digitalgibberish.com/llms.txt.
llms.txt vs. robots.txt vs. sitemap.xml
These three files get confused, and they do different jobs:
| File | Who it’s for | What it does |
|---|---|---|
| robots.txt | Crawlers | Says which parts of your site they may or may not visit |
| sitemap.xml | Search engines | Lists every page you want found |
| llms.txt | AI systems | A short, curated guide to your most useful pages |
llms.txt doesn’t block anything and doesn’t grant permission. If you want to control which AI crawlers can visit, that’s robots.txt. For example, OpenAI’s crawler documentation explains how to allow or block its search crawler and its training crawler separately.
Does it actually help?
Here’s the honest picture:
- Google says you don’t need it. Its guidance on AI features says you don’t need to create new machine-readable files, AI text files or markup to appear in AI Overviews and AI Mode.
- No one publishes a list of which AI systems read it. The llmstxt.org page says agents are expected to look at the file and follow its links, but it doesn’t say which systems do.
- It isn’t a shortcut to being recommended. If your website already has clear pages, sensible headings and accurate information, an assistant can usually figure out what you do without it.
So the impact is small and unproven, and that’s a fair reason to treat it as a bonus and not a priority. What it costs is also small: once your pages are in good shape, writing one is typically a 20-minute job.
Does a small business need one yet?
Not before the basics. In this order, we’d put these first:
- An accurate, complete Google Business Profile
- Clear pages that say what you do, where and for whom
- Matching business details everywhere
- Real reviews
Then, if you like, add an llms.txt file. Our guide to getting ChatGPT to recommend your business covers the bigger levers.
How to add one
You don’t need a developer to write the file, but where it has to go depends on your platform:
- WordPress with Yoast SEO. According to Yoast’s help page, the plugin can generate and refresh an llms.txt file for you. It’s a setting you turn on, not on by default.
- Webflow. It has a help article on uploading an llms.txt file.
- Other platforms. Look for a way to publish a plain text file at the root of your domain, or ask your host or developer. It must end up at yourdomain.com/llms.txt.
After you publish it, open that address in a browser to check that it loads as plain text.
Common mistakes
- Listing everything. It’s meant to be curated. Pick the pages a newcomer would need.
- Vague descriptions. “Our services” tells an AI less than “Emergency plumbing and drain cleaning in Nashua, NH.”
- Never updating it. Links to pages you’ve deleted or renamed make it worse than having none.
- Expecting results. Treat it like housekeeping, not a growth tactic.
Where it fits with everything else
llms.txt is one small piece of what’s sometimes called GEO, or generative engine optimization: making a website easy for AI systems to read and cite, the way SEO makes a website easy for Google to read and rank. We’ve written more about how GEO and SEO relate if you want the fuller picture.
Common questions
What does llms.txt stand for? It’s short for a file for large language models, the technology behind assistants like ChatGPT and Claude. The name is just a convention, like robots.txt.
Will llms.txt improve my Google rankings? Not directly. Google says you don’t need AI text files to appear in its AI features, and it isn’t a ranking factor.
Do ChatGPT, Claude or Gemini read it? We can’t say for any of them. The proposal says agents are expected to look at the file, but no one publishes a definitive list of which AI systems do.
Is llms.txt the same as blocking AI from my site? No. It doesn’t block anything. Use robots.txt for that.
How often should I update it? Whenever you add, remove or rename an important page, and at least a couple of times a year.
Do you need a developer to add one? Not to write it. Putting it in the right place depends on your platform, so ask your host or web person if you aren’t sure.
If you’re not sure whether your current site gives AI assistants, or Google for that matter, enough to work with, that’s exactly what our GEO work looks at: whether your pages, structure and llms.txt file (or lack of one) are helping or hurting. Get in touch if you’d like a second opinion.