llms.txt in Practice: A Site Manual Written for AI

If robots.txt tells crawlers "where you may go," then llms.txt tells AI "what this site is about." It's a Markdown file in your site's root directory that introduces your website to language models in the most compact way possible: who you are, what you offer, where the important content lives. We have one on our own site — this article lays out exactly how to write it.

First, why it's needed. The way an AI engine understands a website is by fetching pages, stripping the HTML, and piecing meaning together from the leftover text. That process is expensive and lossy — navigation bars, footers, and popup text all get mixed in. The logic of llms.txt is direct: instead of letting the AI guess, hand it a manual a human has already organized.

The standard structure of llms.txt

The format was proposed by Jeremy Howard of Answer.AI in 2024. The structure is deliberately simple, written entirely in Markdown, top to bottom:

1. One H1 heading: the name of the site or organization. The only required element in the whole file.
2. One blockquote: a one-to-two-sentence core summary of what the site is. This is the first self-introduction the AI reads — worth polishing repeatedly.
3. A few free-form paragraphs: supporting context — who you serve, areas of expertise, location, contact channels — in plain short paragraphs.
4. Several H2 sections: each section holds a list of links, each link followed by a one-sentence description of the page. Common sections are "Services," "Work," "Articles," "Contact."
5. A special section named Optional: secondary links, explicitly telling the AI "skip these if space is tight."

How we actually wrote ours

Taking our own file as the example, the structure runs roughly: the H1 says "FreeCo"; the blockquote says "a Taiwan-based digital product team building custom websites, e-commerce systems, and AI applications — while also running our own e-commerce and AI tool products." A descriptive paragraph follows on how we work — hand-built to fit, no templates, SEO and GEO built in.

Then the link sections: the "Services" section lists the website, e-commerce, and AI service pages, each link followed by one sentence on "what it offers and who it's for"; the "Articles" section picks a dozen or so pieces that best represent our expertise, rather than dumping in everything; the "Contact" section is a single line pointing to the contact page.

A few lessons from doing it: the one-sentence description matters more than the link itself, because the AI decides whether to fetch further based on that sentence; shorter beats longer — llms.txt is an index, not a content farm, and we keep ours under a hundred lines; write in the reader's language — imagine how a user would ask an AI ("a Taiwan team that builds custom websites") and let your descriptions naturally cover those phrasings.

llms.txt isn't copy written for humans — it's a self-introduction written for machines. What it really tests is whether you can explain who you are in three sentences.

Common questions and practical caveats

"Do AIs actually read it?" Honest answer: it's an emerging convention, not every engine is guaranteed to read it, and adoption is still shifting. But the cost is so low there's almost no reason not to do it — one file, one afternoon. And the exercise itself has value: many clients can't write an llms.txt not because they don't know Markdown, but because they've never thought through "who we are and what we offer" in the first place.

Two more reminders. First, llms.txt needs robots.txt to cooperate — the best-written manual is useless if the crawlers are locked out at the door. GPTBot, ClaudeBot, and PerplexityBot need to be allowed; the full context is in our GEO primer. Second, keep the content in sync with the site — an llms.txt pointing at dead pages is worse than none at all. We've made it part of our checklist for every site revision.

If you want to see a working example, just open the llms.txt at the root of our website — for every brand website project we build, this file is now part of the standard deliverable. A manual written for AI is, in the end, your understanding of your own business translated into a shape machines can read.