Skip to content
Lastmod

Guides

llms.txt explained: what it is and whether you need one

By Yasin Kocak4 min readPublished

The short answer

Llms.txt is a proposed Markdown file at /llms.txt that gives AI assistants a short description of your site and links to your most useful pages. It is not a standard, and the evidence that AI systems fetch or use it is limited. Adding one is cheap and harmless, but it is an experiment, not a way to get mentioned.

What it is

The proposal dates from 2024. The idea is a readable map for language models: a title, a one-paragraph summary and a list of links with a short note on each. Assistants have limited room to read a page, and a clean list of your best pages is easier to use than a website full of navigation and scripts.

It is a convention that people proposed, not a standard backed by a standards body or by the search engines.

How it differs from robots.txt and a sitemap

Robots.txt tells crawlers what they may fetch. A sitemap lists the addresses of all the pages you want search engines to know about. Llms.txt is neither of those: it does not grant or deny access, and it is not a complete list. It is an editorial selection with a short description, written for a reader.

Access is still decided in robots.txt. A bot that is blocked there is not helped by a page that lists your content.

What the file looks like

Plain Markdown. A heading with the name of the site, a short quote block that says what the site is, then headings for groups of pages with a list of links under each. One line per link: the title, the address and a sentence on what the page covers.

  • A first-level heading with the name of the site.
  • A short summary of what the site is and who it is for.
  • Second-level headings for groups, such as product, guides and company.
  • Under each, links with a one-line description.

Where the evidence stops

What the evidence says

Several tools and a few sites support the file, and some assistants will read it when a user points them to it. The large search engines have not said that they use it for ranking or for choosing sources, and no study shows that having one raises how often a site is named.

If someone tells you llms.txt will get you into AI answers, ask for the measurement. The honest position is that it costs little to write and that nobody can show it works.

Should you add one?

If your site is a product with documentation, a developer audience or a set of guides, it takes under an hour and keeps your best pages in one place that a person can also read. Do it if you would be glad to have that list anyway. Do not spend a week on it, and do not put it ahead of fixing crawl blocks, a broken sitemap or thin pages.

Generate it from the same data as your pages if you can, so it does not drift out of date. A file that links to pages that moved is worse than none.

Measure before you believe

If you add one, note the date and watch your AI visibility over the following weeks the way you would for any other change: counts of answers that name you, and visits from assistants. Expect noise. If nothing moves, you have learned that it did not matter for you.

In Lastmod

Lastmod does not fetch or assess the llms.txt of your site and makes no claim about what it does for AI visibility. It does read your robots.txt and your sitemap, which decide access and discovery. Lastmod publishes an llms.txt for its own site, built from the same page descriptions the site already shows.

See it on your own site

Start the 30-day free trial. Lastmod reads your Search Console data and shows what changed, with how sure it is.

30-day free trial · no credit card · database and hosting in the EU