llms-txt-init

Scaffold an llms.txt and an AI-crawler-aware robots.txt for your website in one command.

As AI answer engines (ChatGPT, Claude, Perplexity, Google AI Overviews) become a real source of traffic, two files help them use your content well: a robots.txt that explicitly allows the AI retrieval crawlers you want, and an llms.txt that gives them a clean, plain-text summary of your site. This tool writes both, with sensible defaults.

Usage

No install needed:

npx llms-txt-init --name "My Site" --url https://example.com --description "One line about the site"

Or install globally:

npm install -g llms-txt-init
llms-txt-init --name "My Site" --url https://example.com --description "One line about the site"

Flags

| Flag | Description | | --- | --- | | --name <text> | Site name (required) | | --url <url> | Canonical site URL (required) | | --description <text> | One-line summary (required) | | --sitemap <url> | Sitemap URL (default: <url>/sitemap.xml) | | --out <dir> | Output directory (default: current directory) | | --block-training | Also block training-only scrapers (CCBot, Bytespider, meta-externalagent) | | --force | Overwrite existing files | | -h, --help | Show help |

What it writes

robots.txt — allows the major AI retrieval crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, Applebot-Extended, Amazonbot) so your pages can be cited in AI answers. With --block-training, it also disallows training-only scrapers. Ends with your sitemap line.

llms.txt — a plain-text summary of your site (name, description, key pages, attribution note) that AI engines can retrieve easily.

Review both files, then deploy them to the root of your site (so they're served at /robots.txt and /llms.txt).

License

MIT

Built by Hirak Banerjee. More on AI tools and SEO/GEO at stackwrite.com.