llms.txt: The Guide to Getting Read (and Cited) by AI

What is the llms.txt file?

The llms.txt file is an emerging convention: a Markdown file placed at the root of a site (your-site.com/llms.txt) that gives a machine-readable summary of the site and its priority pages. Its goal is to point AI crawlers toward your key content. It is not an official standard.

The convention was proposed on September 3, 2024 by Jeremy Howard (Answer.AI) on llmstxt.org. The idea behind it is simple: language models have limited context windows, and a page's HTML (menus, scripts, ads) is noisy. A short, clean, well-organized file helps an AI understand your site without having to digest all the code.

In practice, llms.txt is neither a format required by any engine nor a guarantee of being read. It is a community proposal, adopted by some publishers and tools, and it remains experimental. Treat it as a sound organizational practice, not a magic button for AI visibility.

Is llms.txt mandatory for GEO?

No. No search engine or AI assistant requires an llms.txt file today. It is not an official standard, and not having one does not penalize you. Above all, it is no substitute for the essentials: a site whose pages are accessible to AI crawlers, readable without JavaScript and clearly structured in HTML.

With a topic this heavily discussed, the risk is getting your priorities backwards. A perfect llms.txt on a site that crawlers cannot explore, or whose content only appears after JavaScript runs, is useless. The question "do I have an llms.txt?" should never come before "can an AI actually read my pages?".

Key takeaway: llms.txt is optional and complementary. It sits on top of a site that is already accessible and well structured; it never makes up for a site closed to crawlers or dependent on JavaScript. Build the foundations first, add the llms.txt afterward.

What is llms.txt actually for?

It exists to steer LLM crawlers toward your priority pages and hand them a clean, machine-readable summary of your site. Rather than letting an AI guess what matters across dozens of noisy pages, you present a structured map: here is who we are, here is our reference content.

In practice, its most credible uses are the following:

Prioritizing your key pages. You flag your guides, your documentation and your reference product pages, set apart from the rest.
Providing a clean summary. A short, factual text describes your business, free of the noise of HTML, menus and scripts.
Helping the tools that read it. Some coding assistants and agents already read an llms.txt provided explicitly to work on a documentation set. Since November 2024, Mintlify automatically generates an llms.txt for the documentation it hosts.

The benefit is therefore mostly one of clarity and organization. That is useful, but do not mistake it for an instruction that every major engine automatically follows when deciding whom to cite.

How do you create an llms.txt file?

Create a Markdown file named llms.txt and place it at the root of your domain (your-site.com/llms.txt). The conventional structure is simple: an H1 with the site name, a summary as a blockquote, then ## sections listing your key pages as annotated Markdown links.

Here is a minimal example, based on the one GEO console publishes at /llms.txt:

/llms.txt

# GEO console

> GEO console (geo-search-console.com) is the "search console for AI":
  it measures whether a brand is cited by AI assistants — ChatGPT,
  Gemini, Claude, Perplexity and Grok.

## Product
- [Features](https://geo-search-console.com/features): citations, position and share of voice.
- [How it works](https://geo-search-console.com/how-it-works): the measurement method.

## Resources
- [GEO Guide](https://geo-search-console.com/guide): the reference guide to GEO.
- [Blog](https://geo-search-console.com/blog): articles on AI visibility.

Three rules hold it together: keep it short and factual (the file should fit in a limited context window), keep it up to date when your pages change, and keep it consistent with your visible content — an llms.txt that describes something other than your real site undermines your credibility.

llms.txt vs robots.txt vs sitemap.xml: what's the difference?

All three files live at the site root, but they play different roles. robots.txt tells crawlers what they are allowed to explore. sitemap.xml lists your URLs to ease indexing. llms.txt offers an editorial summary of your key pages for LLMs. They complement one another: none replaces the others.

File	Role	Audience	Mandatory?
robots.txt	Allow or block crawling of certain parts of the site	Search and AI crawlers	No, but near-universal and respected
sitemap.xml	List the URLs to index and how fresh they are	Search engines (indexing)	No, but a widely supported standard
llms.txt	Summarize and prioritize key pages for LLMs	Language models and AI agents	No, an emerging convention, not a standard

The lesson is clear: robots.txt and sitemap.xml remain your technical priorities. First confirm that AI crawlers are allowed and that your URLs are properly exposed. llms.txt is an editorial layer on top, not an alternative to those two proven files.

Does llms.txt really improve your AI citations?

Honestly: on its own, nothing proves it so far. The major assistants do not document any use of llms.txt to decide whom to cite. Its effect on your citations in ChatGPT, Gemini or Perplexity remains marginal and unproven. It is not a standalone visibility lever — at best, it is a tidy complement.

The real GEO levers are elsewhere, and they are well documented. On Google's side, the official documentation notes that no structured data or special setup is required to appear in AI features: what counts is useful, accurate and accessible content. In April 2025, Google's John Mueller even publicly compared llms.txt to the keywords meta tag, with no AI service on record using it (Search Engine Journal, April 17, 2025).

Focus your energy on what actually carries weight:

Accessible content

AI crawlers allowed, content readable without JavaScript, fast pages. If the crawler cannot read your page, no llms.txt will get it cited.

A clear structure

Question headings, a self-contained answer paragraph, lists and tables. Scannable content is content that is easy to cite.

Authority

An identified author, real expertise, sourced and dated facts. Trust is the most decisive signal for an AI.

Third-party mentions

Reviews, forums, press, serious directories. A brand cited positively elsewhere is easier to recommend.

To dig into these levers, read our full GEO guide and our article on how to get cited by ChatGPT, Gemini and Perplexity. llms.txt has its place there — small, complementary, never central.

How do you measure the effect?

By comparing your citations before and after. Rather than assuming an llms.txt "helps", measure it: how often the AIs cite you, in what position, and with what share of voice against your competitors. If nothing moves after you add the file, you will know the effort belongs elsewhere.

The problem is that every assistant answers differently, changes often, and never tells you when it starts or stops citing you. Manually testing your customers' real questions on ChatGPT, Gemini, Claude, Perplexity and Grok is doable once, but unmanageable over time.

That is exactly what GEO console, the "search console for AI", does: it automates those tests on ChatGPT, Gemini, Claude, Perplexity and Grok, and tracks over time whether, where and how your brand is cited, along with your share of voice. It is the only honest way to know whether an optimization — llms.txt included — changes anything. To go further, read how to know if your brand is cited.

Without measurement, llms.txt remains a belief. With a before/after measurement on each engine, it becomes a hypothesis you can confirm or discard. That discipline — structure, measure, fix — is what separates serious GEO from guesswork.

Sources

Original llms.txt proposal, llmstxt.org (Jeremy Howard, Answer.AI, September 3, 2024).
Google Search Central, documentation on AI features (no special structured data required).
Search Engine Journal, Google Says LLMs.txt Comparable To Keywords Meta Tag (John Mueller, April 17, 2025).
Mintlify, announcement of automatic llms.txt generation (November 20, 2024).

Frequently asked questions

Is the llms.txt file mandatory?

No. No search engine or AI assistant requires the llms.txt file today. It is an emerging convention, not an official standard. It does not replace an accessible site: if your pages are not readable by AI crawlers, without JavaScript and well structured, an llms.txt will not change anything.

Where do you place the llms.txt file?

At the root of your domain, at your-site.com/llms.txt, exactly like robots.txt. That is the location the convention expects, so the tools that look for it can find it. The file is written in Markdown and should remain readable by both humans and machines.

Will llms.txt improve my citations in ChatGPT or Perplexity?

Nothing proves it so far. The major assistants do not document any use of llms.txt to decide whom to cite. The real levers remain accessible, factual content, a clear structure, your authority and your third-party mentions. llms.txt is at best a tidy complement, never a shortcut to citation.

Does llms.txt replace robots.txt and sitemap.xml?

No, the three files have distinct roles. robots.txt indicates what crawlers may explore; sitemap.xml lists your URLs for indexing; llms.txt offers an editorial summary of your key pages for LLMs. They complement each other, but none substitutes for another.

Do you need an llms.txt if your site is small?

It is not a priority. On a small site that is already clear and well structured, the expected gain from an llms.txt is marginal. First make sure AI crawlers have access, pages are readable without JavaScript and content is factual. llms.txt comes afterward, as an organizational layer, not a prerequisite.

More questions? See our full FAQ or the complete GEO guide.