llms.txt
Also known as /llms.txt · LLM-friendly index
A Markdown file at /llms.txt that gives LLMs a curated, plain-language map of your site and links to its most important content.
What it is
llms.txt is a proposed convention: a Markdown file served at the site root (/llms.txt) that summarizes what the site is about and lists key URLs an LLM should read, often as clean Markdown. An optional /llms-full.txt can inline the full expanded content. It is an index for machines, not a crawler directive like robots.txt.
Why it matters
AI assistants and answer engines have limited context windows and cannot crawl an entire site at inference time, so a concise, authoritative index helps them find and quote the right pages. A good llms.txt increases the chance your facts, docs, and product pages are surfaced accurately in AI answers rather than guessed or skipped.
How to verify
Run curl -sI https://example.com/llms.txt to confirm it returns HTTP 200 with a text/plain or text/markdown content type, then curl -s https://example.com/llms.txt to inspect the content. Check that the file is valid Markdown, starts with an H1 title and short summary, and that the linked URLs resolve.
How to fix
Create a Markdown file with an H1 site name, a one-paragraph summary, and sectioned lists of links (Docs, Products, Policies) each with a short description, then deploy it at /llms.txt. Keep it short and current, and optionally add /llms-full.txt with the expanded text for models that want the full corpus.
Related terms
- AI-crawler policyExplicit robots.txt rules for AI user agents (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) that deliberately allow or block them.
- Markdown content negotiationServing a clean Markdown representation of a page when a client requests it via the Accept: text/markdown header.
- XML SitemapAn XML file listing a site's canonical URLs to help search engines discover and prioritize pages for crawling.
- Entity ClarityStating plainly who the brand is, what it does, and its key facts so people and machines can identify the entity without ambiguity.
Official references
External, opens in a new tab.
Put this into practice.
Work through every check by hand and turn it into a shareable GEO Score report — or scan your site automatically in seconds.