llms-txt-kit

Zero-dependency CLI that generates and validates llms.txt files — the emerging standard that tells AI assistants and answer engines what your site is about. Point it at your site (or sitemap) and get a clean, spec-shaped llms.txt; point it at an existing file and get a lint report with fix hints and CI-friendly exit codes.

Quick start

Node.js 20+ is the only requirement — no npm install needed.

git clone https://github.com/madahzadeh/llms-txt-kit.git
cd llms-txt-kit

node llms-txt.mjs generate https://example.com/ --out llms.txt   # crawl and generate
node llms-txt.mjs generate --sitemap https://example.com/sitemap.xml --out llms.txt
node llms-txt.mjs validate https://example.com/llms.txt --check-links
npm run demo   # offline demo: generate + validate against a built-in fixture site
npm test       # deterministic test suite

What generate does

Crawls your site (or reads your XML sitemap, sitemap-index included), extracts each page's <title> and meta description, groups pages into sections by URL path, and emits a spec-shaped file:

# Acme Tools

> Small, fast developer tools for busy teams.

## Docs

- [Getting started](https://example.com/docs/getting-started): Install and run your first audit in one minute.
- [API reference](https://example.com/docs/api): Every endpoint, flag, and exit code.

Review and edit the result before publishing — the generator gives you a correct skeleton, you supply the editorial judgment.

What validate checks

| Code | Check | Meaning | |---|---|---| | V001 | missing-h1 | file must start with a single # Title | | V002 | missing-summary | no > summary blockquote after the title | | V003 | invalid-link-item | list item is not - [name](url): description | | V004 | relative-url | link URLs must be absolute | | V005 | duplicate-url | the same URL listed twice | | V006 | broken-link | listed URL returns ≥ 400 (with --check-links) | | V007 | empty-section | a ## section with no links | | V008 | multiple-h1 | more than one H1 | | V009 | optional-not-last | ## Optional must be the last section |

CLI options

| Option | Default | Description | |---|---|---| | --sitemap <url> | — | generate from a sitemap instead of crawling | | --out <file> | stdout | where to write the generated file | | --max-pages <n> | 100 | generation page limit | | --concurrency <n> | 5 | parallel requests | | --timeout <ms> | 10000 | per-request timeout | | --check-links | off | validate: fetch every listed URL | | --fail-on <level> | error | error | warning | none — controls exit code | | --json | — | machine-readable JSON report (validate) | | --no-color | — | disable colored output |

Exit codes: 0 clean · 1 findings at/above --fail-on · 2 usage/runtime error.

CI example

name: llms-txt
on:
  schedule:
    - cron: "0 6 * * 1"
  workflow_dispatch:

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: |
          curl -sL https://raw.githubusercontent.com/madahzadeh/llms-txt-kit/main/llms-txt.mjs -o llms-txt.mjs
          node llms-txt.mjs validate https://example.com/llms.txt --check-links --no-color

Limitations

Honest scope: the generator uses titles and meta descriptions from server-rendered HTML — JavaScript-rendered pages are not executed. Section grouping is by first URL path segment, which fits most sites but is a heuristic. llms.txt itself is an emerging convention: publishing one makes your site easier for AI assistants to summarize accurately, but no engine guarantees consumption.

Hire me

I build AI automation, release workflows, mobile products, and technical-SEO-driven web systems for founders and international businesses.

• Upwork: madahzadeh.com/upwork
• Portfolio and contact: github.com/madahzadeh · iequity.co