Skip to content

Glossary · Technical

Definition

llms.txt is a proposed convention for a plain Markdown file at the root of a website (/llms.txt) that gives large language models a concise summary of the site and links to its most useful pages. It is optional, not an official standard, and there is no confirmation that Google uses it.

  • Also: llms txt file
  • Also: LLMs.txt standard
  • Also: llms-full.txt
  • Also: AI sitemap

How llms.txt works

The idea was proposed in 2024 to solve a practical problem: web pages are full of navigation, scripts and adverts, and AI tools working within limited context struggle to find the substance. An llms.txt file offers a clean map instead, with a title, a short description and grouped lists of links, sometimes pointing to Markdown versions of key pages. A companion llms-full.txt can hold the full text.

It differs from robots.txt, which tells crawlers what they may access, and from an XML sitemap, which lists URLs for indexing. llms.txt is guidance about what matters, written for language models. Some documentation platforms generate one automatically, and some AI coding tools read them when pointed at a site.

The misconception is that llms.txt makes a site visible in AI answers. No major AI search product has confirmed relying on it for ranking or citation, and Google has given no indication that it does. Treat it as a cheap, low-risk addition, not a substitute for crawlable pages and clear content.

How to improve it

  1. 01Fix crawl access, indexing and page clarity first, since these matter far more.
  2. 02If you add the file, start with a one-paragraph summary of who you are and what you offer.
  3. 03Group links to your most useful pages under short headings, with a one-line note on each.
  4. 04Keep it accurate as pages change, because an outdated map is worse than none.
  5. 05Do not use it to block or allow crawlers; that remains the job of robots.txt.
An example

A payroll software company with hundreds of help articles adds an llms.txt listing its setup guides, integration pages and pricing explanation in plain sections. Its developers already point AI coding tools at the file. The marketing team records it as an optional addition, not a visibility tactic.

Questions

Questions about llms.txt

Still unsure? Ask the team or write to support@deylaxy.com.

Does Google use llms.txt?

There is no confirmation that Google uses llms.txt for search, AI Overviews or ranking. Google's guidance for appearing in its AI features points to standard SEO fundamentals: crawlable, indexable, helpful pages. Adding the file is unlikely to cause harm, but it should not be expected to change how Google treats your site. Its main uses so far are with documentation tools and some AI assistants.

Is llms.txt the same as robots.txt?

No. Robots.txt is an established standard that tells crawlers which parts of a site they may access, and well-behaved bots follow it. llms.txt is a proposed, optional file that summarises a site and links to its key content for language models. It grants or blocks nothing. If you want to control AI crawler access, you still need robots.txt rules.

Related

Related terms

Get your free Visibility Intelligence Audit

A concise reading of verified signals across search engines and AI answers, enough to recognise the gaps, not a playbook for competitors.