llms.txt: The File That Helps AI Understand Your Website
What is llms.txt and why is everyone talking about it?
llms.txt is a plain-text file written in Markdown that you place at the root of your website to tell large language models (ChatGPT, Claude, Gemini, Perplexity) which pages matter most and how they should be interpreted. Think of it as a curated menu for AI: instead of letting a model crawl every URL on your site, you point it straight to the content you want it to read.
The proposal came from Jeremy Howard, co-founder of Answer.AI and fast.ai, in late 2024. His logic was simple: traditional search engines have robots.txt to know what to crawl, so language models need something equivalent, but built around how they actually process information.
Key data
A SE Ranking study of nearly 300,000 domains found that only 10.13% of websites have an llms.txt file in place. Nine out of every ten businesses still haven't shipped one.
Why does this matter for your business? Because 40% of consumers already use AI tools to discover products and local services. If your competitor has an llms.txt and you don't, you're handing them a free advantage. For the wider picture of how AI is reshaping search, start with our guide on what GEO is.
How is llms.txt different from robots.txt?
llms.txt doesn't replace robots.txt. They're complementary tools with opposite purposes. Robots.txt tells crawlers what they shouldn't touch. llms.txt tells AI what content it should read first. One excludes, the other invites.
The core difference is who reads them and how. Robots.txt was created in 1994 for search engines like Google and Bing, and uses technical directives (Allow, Disallow). llms.txt was designed for language models that process natural text, which is why it's written in Markdown.
As Search Engine Land puts it, the best analogy is this: robots.txt is a "do not enter" sign, while llms.txt is a treasure map.
| Feature | robots.txt | llms.txt |
|---|---|---|
| Year created | 1994 | 2024 |
| Format | Plain text with directives | Structured Markdown |
| Goal | Control crawler access | Guide AI to key content |
| Audience | Googlebot, Bingbot, etc. | ChatGPT, Claude, Gemini, Perplexity |
| Main action | Block paths | Recommend pages |
| Official standard | Yes (IETF RFC 9309) | No (community proposal) |
What about sitemap.xml?
It's also worth separating it from sitemap.xml. The sitemap lists every URL on your site so search engines can discover them. llms.txt is selective: it only includes the pages you actually want AI to use as a source. Less is more.
The ideal setup uses all three: robots.txt for access control, sitemap.xml for full indexing, and llms.txt for pointing AI to your best content.
What does an llms.txt file look like?
The structure is simple Markdown with four main parts: an H1 with your company name, a short summary in a blockquote, optional sections with relevant context, and links to the key pages on your site.
You don't need to write code. If you can write an email with bullet points, you can write an llms.txt. Here's an example for a SaaS documentation site:
# Acme Docs
> Acme is a developer platform for shipping background jobs at scale.
> Our docs cover quickstart guides, the REST API, SDKs in five languages,
> and production playbooks for teams running 100M+ jobs per month.
## Getting started
- [Quickstart in 5 minutes](https://acme.com/docs/quickstart.md)
- [Core concepts](https://acme.com/docs/concepts.md)
- [Authentication](https://acme.com/docs/auth.md)
## API reference
- [REST API](https://acme.com/docs/api.md)
- [Webhooks](https://acme.com/docs/webhooks.md)
- [Rate limits](https://acme.com/docs/limits.md)
## Guides
- [Production checklist](https://acme.com/docs/production.md)
- [Migrating from cron](https://acme.com/docs/migrate.md)
The key parts
- H1 title: the name of your business or product. This is the only required section
- Blockquote: a 2-3 sentence summary explaining what you do and why you matter
- H2 sections: group your links by category (getting started, API, guides, FAQ)
- Markdown links: each link ideally points to a Markdown (
.md) version of the page, which is easier for AI to parse
One detail worth knowing: links can point to .md versions of your pages or to the normal URLs. The original spec recommends Markdown versions, but HTML works too. Companies like Mintlify, Anthropic and Vercel publish both formats side by side.
Why should a small business care?
For a small business, llms.txt is a chance to get ahead before your competitors notice. Implementing it takes less than an hour, costs nothing, and positions your site as ready for the AI-driven search era.
Search is shifting fast. According to Hostinger, the LLM market will hit $105.5 billion by 2030. More people now ask ChatGPT or Gemini instead of typing into Google.
If your site isn't set up for these models to understand it, you're missing out on a channel that's growing exponentially.
Concrete benefits
- Control over your narrative: you decide what AI sees rather than letting it guess
- Priority reading: the model goes straight to your key pages instead of getting lost in menus and sidebars
- Differentiation: if 90% of websites still don't have llms.txt, you move ahead by default
- Zero cost: it's a text file you can write and upload in minutes
- Compatible with current SEO: it doesn't replace anything you already do, it just adds a layer
Key data
73% of marketing professionals believe generative AI will reshape how consumers discover brands, according to McKinsey. Preparing your site today is investing in tomorrow's visibility.
How do you create an llms.txt file step by step?
Building an llms.txt takes three steps: write the content in Markdown with the right structure, save it as llms.txt, and upload it to your root domain so it's reachable at yourdomain.com/llms.txt.
Step 1: Choose your key pages
Before writing anything, ask yourself:
- Which 5-10 pages best describe what my business does?
- What questions do I get most often from customers or prospects?
- What content would I want AI to cite when someone asks about my services?
Don't try to cram your whole site in. The point of llms.txt is to curate, not to list. Think about the pages a journalist would need to read before writing about your business.
Step 2: Write the file
Open any text editor (Notepad on Windows, TextEdit on Mac, or VS Code). Follow this template:
- Write your business name with a
#in front (H1) - Underneath, add a 2-3 sentence summary inside a
>blockquote - Create sections with
##to group your links - List each page with a dash and a Markdown link:
[text](URL)
Save the file with the exact name llms.txt.
Step 3: Upload it to your site
The file has to be reachable at the root of your domain. So if your site is www.yourshop.com, it should be readable at www.yourshop.com/llms.txt.
- WordPress: use a plugin like "Add Custom File" or upload via FTP to the root folder
- Shopify: go to Settings > Files and upload it as a static asset
- Webflow: the platform already includes native support for llms.txt
- Custom site (Next.js, Astro, etc.): drop the file into the
public/folder or the server root
Step 4: Verify it works
Open your browser and visit yourdomain.com/llms.txt. You should see the file contents in plain text. If it doesn't load, check the file location and your server permissions.
Can llms.txt be combined with other AI visibility tactics?
Yes, and it should be. llms.txt works best as part of a complete AI optimization strategy that includes schema markup, structured content, and monitoring how the different language models mention your brand.
llms.txt is one piece of the puzzle, not the whole solution. To maximize your visibility, pair it with:
- Schema markup: add structured data to your pages so AI understands your content at a technical level. Our schema markup for AI guide covers the implementation
- FAQ content: build question-and-answer sections that LLMs can extract directly
- Citations and sources: include data, stats and references that give your content credibility
- Up-to-date company profile: keep contact info, addresses and service descriptions current
| Action | Effort | AI impact | Priority |
|---|---|---|---|
| Schema markup | Medium | High (3.7x more citations) | High |
| llms.txt | Low | Medium-high (future-proofing) | Medium-high |
| FAQ content | Low | High (3.2x in AI Overviews) | High |
| Directory profiles | Low | Medium | Medium |
| Visibility monitoring | Low | High (lets you iterate) | High |
Are there any limits or risks?
The main limitation is that no AI provider (OpenAI, Google, Anthropic) has officially confirmed they use llms.txt during crawling. It's a community proposal, not a ratified standard. But implementing it carries no risk: it can't hurt your site.
Be honest here. According to a Search Engine Journal analysis, there's no proven correlation between having an llms.txt and getting more citations from AI models. Google has compared it to the old keywords meta tag, which stopped having any effect years ago.
What we know for sure
- No downside: adding llms.txt has zero negative effect on your SEO or site performance
- Free and fast: less than an hour of work to implement
- Adoption is rising: from 5% in 2025 to 10% in 2026, with an upward trend
- Major companies use it: Anthropic, Vercel, Cursor, Mintlify and hundreds of open source projects already have it
What we still don't know
- Whether LLMs actively read it during inference (when generating answers)
- When it will become an official standard
- Which platforms will integrate it into their crawlers
The pragmatic call: ship it because the cost is near zero and the upside is real. It's like putting your business on Google Maps when it just launched. Not everyone did it, but the early movers won.
How do I know if AI is already citing my site?
To find out whether your site shows up in ChatGPT, Claude, Gemini or Perplexity answers, you need a tool that runs real queries against these models and tracks whether your brand is mentioned, how often, and where you sit versus competitors.
That's exactly what Surfeo does. It runs audits against the main AI models, calculates a visibility score, and shows you specific recommendations to improve your position. You can measure the real impact of actions like adding llms.txt or schema markup.
Because preparing your site for AI is pointless if you can't tell whether it's working. Run a free visibility test to see your starting point.
Is llms.txt worth implementing in 2026?
Yes. The llms.txt file is a one-hour investment that positions your site for the future of AI-powered search. No cost, no risk, and it gives you control over how language models interpret your business.
The situation looks a lot like the early days of SEO. Twenty years ago, plenty of businesses didn't see the point in optimizing for Google. Today, the ones that skipped it regret it. Generative AI is following the same script: the time to prepare is now, not when everyone else already has.
Three things you can do today:
- Create your llms.txt following the structure in this article
- Add schema markup to your main pages to give AI technical context
- Measure your current visibility with a tool like Surfeo so you know your baseline
90% of websites still don't have an llms.txt file. Every day you wait is a day your competition can move first.
Keep reading
- How to appear in ChatGPT — A complete playbook for showing up in ChatGPT answers.
- SEO vs GEO — Where traditional SEO ends and generative engine optimization begins.
- Creating content for AI — Writing patterns that LLMs actually pick up and cite.
