Яндекс.Метрика

llms.txt and llms-full.txt: what they are and whether you need them

llms.txt is a Markdown file at the site root: a name, a short description and a list of key pages with links. Jeremy Howard proposed it in September 2024. The file is cheap, but do not expect much from it.

Why it matters

  • A short list of what matters is easier for an agent than crawling menus and footers.
  • Real requests are rare. According to Ahrefs, 97% of sites with a valid llms.txt got zero requests to it in a month, and Google Search does not use it. That is why llms.txt weighs 3 out of 100 in our score.
  • For API documentation the file is more useful: coding agents read it noticeably more often.

What to do

  1. 1

    Follow the format

    An H1 with the name, a quote with a one- or two-sentence description, H2 sections with links and notes. Link to .md versions of pages if you have them.

    # Product name

> What it is and who it is for, in one or two sentences.

## Product
- [Features](https://example.com/features.md): what it does
- [Pricing](https://example.com/prices.md): prices and limits

## Developers
- [API](https://dev.example.com/): REST API, token auth
  2. 2

    llms-full.txt depends on the site

    Documentation benefits from the full text in one file. For a marketing site, redirecting /llms-full.txt to the Markdown version of the homepage is enough, and we count that as a pass.

  3. 3

    Serve a file, not HTML

    A common mistake: an SPA returns its HTML shell with a 200 on /llms.txt. For an agent that is not a file.

    curl -sI https://example.com/llms.txt | grep -i content-type

What the scanner checks

  • llms.txt exists

    A short file for language models: what the product is and where the key pages are.

  • llms-full.txt exists

    The full text of the site or docs in one file. An agent reads it in a single request.

  • llms.txt is structured well enough to navigate

    A heading, sections and at least five links. A one-line file does nothing for an agent.

Check your own site

Paste the address and get the same breakdown in half a minute.

Check a site →

Sources

Other guides