CiteSiteFree tool
llms.txt generator
Reads your site, groups what it finds, and writes a draft llms.txt in the conventional format — name, definition, then linked sections. You edit it and publish it.
Produces a draft you edit, not a file to publish unread. Which pages matter is a judgement about your business, and a crawler does not have one.
What the file is for
Everything a language model knows about your site, it has inferred. It read your navigation, your hero copy, a pricing table and a footer, and it worked out what you sell from the arrangement. That inference is usually right and occasionally, expensively, wrong — two companies with similar names merge into one, a discontinued product stays in the summary, a positioning line from a page you rewrote two years ago survives.
llms.txt is the flattest available answer to that. One file, no navigation, no styling, no marketing structure: this is the name, this is the sentence, these are the pages that matter and what each one contains.
Be honest about the evidence
No engine has published a commitment to read llms.txt at that path. Anyone selling it as a ranking factor is ahead of what is known, and CiteSite is in the business of saying which of its numbers are measured and which are inferred, so it does not get to make an exception here.
The case that stands up is quieter. It is a page on your site. It is crawled like any other page. It states your entity, your scope and your key URLs in a form that costs a reader nothing to parse. It takes twenty minutes. The downside is a text file nobody reads; the upside is that the one thing you most want understood is written somewhere unambiguous.
What this generator does, exactly
- Fetches up to fourteen pages of your site, obeying robots.txt.
- Takes the name from your schema.org markup where you have it,
og:site_namewhere you do not, and the homepage title after that. - Groups pages into Documentation, Product, Writing, Company and Pages by path, and files anything ambiguous under Pages rather than guessing.
- Uses each page’s own meta description, falling back to its first heading and flagging that it did.
- Tells you when your robots.txt would stop an engine reading the file anyway — which is the finding that matters more than the file.
If it could only read one or two pages, that is itself the result worth having: the engines see roughly what this crawler saw, and a site that renders its content in the browser rather than in the response has a bigger problem than a missing text file. Check your crawler access before spending time on this one.
Questions
- What is llms.txt?
- A plain-text file at the root of a site — /llms.txt — that says what the site is and links to the pages carrying the substance. The convention is an H1 with the name, a blockquote with a one-sentence definition, then Markdown sections of links, each with a short note on what it contains.
- Do the AI engines actually read it?
- No engine has committed to it, and you should treat anyone claiming otherwise with suspicion. What is true is narrower and still useful: it is a page on your site, it is fetched by crawlers like any other, and it states your entity, your positioning and your important URLs in the flattest possible form. That is worth having whether or not a named consumer ever asks for it by path.
- Is this a replacement for a sitemap?
- No, and they are not comparable. A sitemap is a machine-readable list of every URL, for a crawler deciding what to fetch. llms.txt is a short, opinionated, human-written summary for a reader deciding what matters. A sitemap with twelve entries would be broken; an llms.txt with four hundred is useless.
- Why is this a draft rather than a finished file?
- Because the one judgement a crawler cannot make is which pages matter to your business. This tool reads your site, groups what it finds and writes the file in the right shape — that is the tedious nine-tenths. Choosing what stays is yours, and a generator that pretended otherwise would ship a confident file listing the wrong pages.
- Does it use AI to write the descriptions?
- No. Every description is the page's own meta description, or its first heading where it has none, and lines that fell back to a heading are flagged. Nothing is summarised or invented — a plausible sentence about a page we half-read is exactly what a file claiming to be authoritative must not contain.
Related
llms.txt: what it is, what it is not, and whether you need one — the format in full, the claims that are not supported, and how to write the one line that matters.
A file describing your site is a claim about how you want to be understood. CiteSite measures the other half: what ChatGPT, Gemini and Perplexity say when someone actually asks.