llms.txt is a proposed plain-text file, written in Markdown and placed at the root of a website (example.co.uk/llms.txt), that gives AI tools a short, curated guide to the site: what it is and which pages matter most. The idea is to help AI systems find and read your key content without wading through menus, scripts and pop-ups.
How llms.txt works
The format was suggested in September 2024 by Jeremy Howard of Answer.AI. It is a proposal, not an official standard, and nothing about it is enforced. A typical file contains:
- A heading with the name of the site or business.
- A one- or two-sentence summary of what it does.
- Sections of links to the most useful pages, each with a short note on what the page covers.
- Optionally, a section of secondary links that a tool can skip when it is short of space.
Some sites also publish a longer companion, often called llms-full.txt, holding the full text of key pages in one place, or offer plain Markdown versions of individual pages.
It is easy to confuse with robots.txt, but the two do different jobs. robots.txt tells crawlers what they may fetch, and reputable crawlers obey it. llms.txt allows and blocks nothing; it is a reading list that a large language model or AI tool may or may not choose to read. In spirit it is closer to an XML sitemap, but written for a model to understand rather than for a crawler to process.
Why it matters
Less than the attention it gets suggests. At the time of writing (October 2026), no major AI search product has publicly committed to reading llms.txt when deciding which sources to cite, and Google staff have said publicly that Google Search does not use it. Adding one will not, on its own, get your business mentioned in ChatGPT, Gemini or Google’s AI features.
Where it does earn its keep is in software documentation. Developer-focused companies publish llms.txt files so people can point a coding assistant at them and get accurate answers about the product. If your business sells software or has extensive help pages, that use is real.
For a typical UK service business or online shop, the file is cheap to create and does no harm, but it is not where AI visibility comes from. Being cited by AI tools depends on the same things as being found in search: pages that answer questions clearly, a site crawlers can reach, and a reputation other sites confirm.
Common mistakes
- Treating it as a ranking switch. Publishing the file and expecting AI mentions to follow.
- Using it to block AI crawlers. It cannot. Blocking is done in robots.txt, using the crawler’s user agent name, such as GPTBot.
- Listing every page. A file that links everything defeats the point; it should be a short, chosen list.
- Letting it go stale. Links to deleted pages, or old prices, in a file nobody remembers creating.
- Neglecting the pages it points to. A tidy llms.txt in front of thin, outdated pages helps nobody.
How to act on it
If you want one, keep it small. Write a heading with the business name and a plain summary, such as “Independent kitchen fitter covering south-west London and Surrey”. Then link your main service pages, prices, contact page and two or three of your most useful guides, each with a one-line description. Upload it to the root of the site, check it loads in a browser, and review it whenever you add or remove a major page.
Then put most of your effort elsewhere: make sure AI crawlers are not blocked by accident, that key pages show their text without relying on scripts, and that each page answers its question directly. That is the substance of generative engine optimisation, and it is how I approach AI search optimisation: llms.txt is a small extra at the end, not the plan.