llms.txt, explained honestly
A small markdown file that tells AI systems what your business is and where the important pages live. It takes about twenty minutes to write, costs nothing, and is worth doing. It is also not the ranking trick some people are selling it as, so here is a straight account of both.
What it actually is
llms.txt is a plain markdown file that sits at the root of your website, at yourdomain.com/llms.txt. It contains a short description of what you do and a tidy list of your key pages with a line explaining each. That is the whole idea. It is a summary written for a machine that reads text, rather than for a human skimming a homepage.
The reasoning behind it is simple. When an AI assistant tries to answer a question about your business, it has to work out what you are from whatever it can fetch. Modern sites make that harder than it sounds: navigation menus, cookie banners, marketing copy and scripts all get in the way of four or five plain facts. A short file that states those facts directly removes the guesswork.
What it does not do
Being straight about this matters, because there are people charging for llms.txt as though it were a ranking lever.
- It is not a ranking factor. Google has not said it uses it. Adding one will not move you up the results page.
- It is not a standard anybody is obliged to follow. It is a proposed convention. Adoption is patchy and voluntary.
- It does not grant or block access. That is
robots.txt. If your robots file blocks AI crawlers, a beautifully written llms.txt changes nothing, because nothing comes to read it. - It will not rescue a site with nothing to say. It summarises what exists. If the underlying pages are thin, the summary is thin.
So why bother? Because it costs twenty minutes and it is one of the few things that makes your business unambiguous to a machine. When an assistant is deciding whether to name you or a competitor, certainty helps. The downside risk is zero and the upside is real, if modest.
Do this first: let the AI crawlers in
This is the part that genuinely matters and the part most sites get wrong by accident. Open your robots.txt and check you are not blocking the assistants you want to be cited by:
GPTBot— OpenAI, feeds ChatGPTClaudeBot— AnthropicPerplexityBot— PerplexityGoogle-Extended— controls use in Google’s AI answers, separately from normal search
Blocking these is a legitimate choice if you do not want your content used for AI answers. Just make it a decision rather than an accident, and know that blocking them means you will not be cited by them either.
How to write one
Keep it factual and short. Every line should be something you would be happy to see quoted back at you, because that is exactly what may happen.
A workable structure
- A title line with your business name.
- A one-paragraph summary as a blockquote: what you do, where, and the two or three things that genuinely distinguish you.
- Contact: phone, email, address if you have premises, opening hours.
- Key pages: a markdown list of links, each with a short explanation of what is on it. Services, prices, locations, contact.
- Important notes: anything a machine could easily get wrong. Which areas you genuinely cover. Whether you charge for a quote. Anything you do not do.
Two rules save you from the common mistakes. Write facts, not marketing, because “the region’s leading provider” is unverifiable and useless to a machine. And never put anything in it that is not on the site itself, since a summary that contradicts your pages is worse than no summary at all.
How to check it is working
- Visit
yourdomain.com/llms.txtin a browser. You should see plain text, not a 404 and not your website design wrapped around it. - Confirm it is served as
text/plainrather than HTML. - Ask an assistant a question about your business and see whether the answer matches your file. If it invents an opening time you never published, you have found a gap worth filling.
- Update it when the underlying facts change. A stale summary is a liability.
Quence checks whether the file exists, whether it is served correctly, and separately whether your robots.txt is letting the AI crawlers reach your site at all. The second check is the one that finds real problems, because a blocked crawler is a silent failure that nothing on your site will ever tell you about.