llms-agent-score

A CLI tool and Node.js library that scores documentation sites for AI-agent friendliness — checking discoverability, structure, link stability, authentication barriers, llms.txt availability, and more.

Built on top of the afdocs scoring engine with an animated terminal UI, rich output formats, and a llms.txt generator.

Demo

Phase 1 — Matrix rain while fetching pages:

  ▓ ░ ▒ 5 ░ ▓ J ▒ ░ 9 ▓ ░ ▒ K ░ ▓ ▒ ░ 3 ▓ ░ ▒ ░ ▓ Z ▒ ░ ▓
  ░ ▒ ▓ ░ Q ▒ ░ ▓ ░ ▒ 7 ▓ ░ ▒ ▓ ░ X ▒ ▓ ░ ▒ ░ ▓ ▒ ░ ▓ ▒ ░
  ▒ ▓ ░ ▒ ▓ ░ ▒ ▓ 2 ░ ▓ ▒ ░ ▓ ░ ▒ ▓ ░ ▒ ▓ ░ R ▒ ▓ ░ ▒ ▓ ░

Phase 2 — Dino running with live progress bar and per-check results scrolling in:

  ✨ pulling up the receipts on developer.atlassian.com

        ████
      ███████
      ██ █████
█  ████████
███████████
    ██  ██
   ███  ███
··· ··  · ··   ·· ···  ·
 ·· ···  · ·  ·· ···
  ··  ···   ·  ·· ···
━━━━━━━━━━━━━━━━━━━╌╌╌╌╌╌╌╌╌╌╌  28/47  14s  60%
⠹  content-structure.heading-hierarchy

  🔍  Discovery Era
    ✓  robots-txt-accessible         Robots.txt found and accessible
    ✓  sitemap-accessible            Sitemap found at /sitemap.xml
    ⚠  llms-txt-accessible          llms.txt not found — consider adding one
    ✓  canonical-urls                Canonical URLs consistent across pages

  📝  Markdown Check (slay or nay)
    ✓  markdown-available            Markdown version available
    ○  markdown-quality              skipped — dependency did not pass

  📏  Size Check (we don't gatekeep)
    ✓  page-size-reasonable          Median page size within limits

  🏗️   Structure Arc
    ✓  heading-hierarchy             Heading hierarchy is consistent
    ✗  structured-data               No structured data found on sampled pages

Phase 3 — Animated score reveal with matrix digit scramble + fill bar:

  ██████████████████████████████░░░░░░░░░░   73/100  B

Phase 4 — Typewriter verdict:

  Your docs score a B (73/100) — solid foundation, a few gaps to close.

Install

npm install -g llms-agent-score

Or run directly without installing:

npx llms-agent-score score https://your-docs-site.com

CLI Usage

score — analyse a docs site

agent-score score <url> [options]

| Option | Description | Default | |---|---|---| | -f, --format | Output format: html, scorecard, json, markdown | html | | -o, --output <file> | Write output to a file | auto | | --show-pages | Show all sampled page URLs | top 10 | | --no-suggestions | Suppress improvement suggestions | — | | --no-drilldown | Suppress per-check failing page lists | — | | --no-impact | Suppress score impact estimates | — | | --max-links <n> | Max pages to sample | — | | --sampling <strategy> | URL sampling: random, deterministic, none | — | | --max-concurrency <n> | Max concurrent HTTP requests | — | | -v, --verbose | Show all failing pages per check | — |

Examples:

# HTML report (auto-opens in browser)
agent-score score https://docs.example.com

# Scorecard to terminal
agent-score score https://docs.example.com --format scorecard

# JSON output piped to a file
agent-score score https://docs.example.com --format json -o report.json

# Lightweight scan (limit pages)
agent-score score https://docs.example.com --max-links 20

bulk — score multiple sites and compare

agent-score bulk <url1> <url2> [url3...] [options]

Score multiple documentation sites and compare results side-by-side. By default, each site is scored sequentially with the full animated terminal UI (dino sprite, matrix rain, live progress bar, score reveal). Use --no-animation to score sites in parallel without the animation.

| Option | Description | Default | |---|---|---| | --urls-file <file> | Path to a text file with one URL per line | — | | -c, --concurrency <n> | Max sites scored in parallel (used with --no-animation) | 3 | | -f, --format | Output format: html, scorecard, json | html | | -o, --output <file> | Write output to a file | auto | | --no-history | Do not record runs to history | — | | --max-links <n> | Max pages to sample per site | — | | --sampling <strategy> | URL sampling: random, deterministic, none | — | | --full-scan | Scan all discovered pages (disables sampling) | — | | --no-animation | Disable per-site animated UI and score in parallel instead | — |

Examples:

# Compare two sites — HTML report auto-opens in browser
agent-score bulk https://docs.site1.com https://docs.site2.com

# Terminal comparison table
agent-score bulk https://docs.site1.com https://docs.site2.com --format scorecard

# Score from a file
agent-score bulk --urls-file sites.txt

# Full scan, save to file
agent-score bulk https://docs.site1.com https://docs.site2.com --full-scan -o report.html

sites.txt format:

# One URL per line, lines starting with # are ignored
https://docs.site1.com
https://docs.site2.com
https://docs.site3.com

What the bulk report shows:

• 🏆 Winner badge on highest scoring site
• Side-by-side score, grade, pass/warn/fail counts
• Category score breakdown — weighted points per category showing why more passes ≠ higher score
• Sortable comparison table (click any column header)
• Per-site collapsible check details
• History recorded per site (foundation for upcoming history command)

Requires at least 2 URLs. Exit code is 1 if any site has failing checks.

generate — create a starter llms.txt

agent-score generate <url> [options]

| Option | Description | Default | |---|---|---| | -o, --output <file> | Output file path | ./llms.txt | | --max-links <n> | Max pages to discover | — |

Example:

agent-score generate https://docs.example.com -o ./llms.txt

Library Usage

Install as a dependency:

npm install llms-agent-score

scoreUrl() — convenience all-in-one

import { scoreUrl } from 'llms-agent-score';

const result = await scoreUrl('https://docs.example.com');

console.log(result.score);        // { total, grade, breakdown }
console.log(result.suggestions);  // ranked improvement suggestions
console.log(result.drilldown);    // per-check failing page lists
console.log(result.sampledPages); // pages that were tested

Individual exports

import {
  runChecks,          // raw afdocs check runner
  computeScore,       // score + grade from a report
  getAllSuggestions,  // ranked suggestions
  getFailingPages,    // per-check drilldown
  extractSampledPages,
  generateLlmsTxt,
} from 'llms-agent-score';

What gets checked

| Category | Checks | |---|---| | 🔍 Content Discoverability | robots.txt, sitemap, llms.txt, canonical URLs | | 📝 Markdown Availability | Markdown versions of pages, quality | | 📏 Page Size | Response size, payload bloat | | 🏗️ Content Structure | Heading hierarchy, code blocks, structured data | | 🔗 URL Stability | Redirects, broken links, versioned URLs | | 👁️ Observability | Last-modified headers, freshness signals | | 🔐 Authentication | Pages behind auth, access barriers |

Exit code is 1 if any checks fail (useful for CI).

Score impact projections

Each suggestion includes a projected score if that check is fixed (e.g. +12 pts → 77 → 89).

Note: Projected scores are per-check estimates and are not additive. Fixing multiple checks may yield less than the sum of individual improvements due to the scoring cap system. Use the projections to prioritise which single check to fix first, not to forecast a combined total.

Output formats

Single-site (score command)

| Format | Description | |---|---| | html | Full report with suggestions, opens in browser | | scorecard | Rich terminal table with score + suggestions | | json | Machine-readable full report | | markdown | Markdown summary report |

Multi-site (bulk command)

| Format | Description | |---|---| | html | Comparison report with category breakdown, opens in browser | | scorecard | Terminal comparison table with 🏆 winner + score breakdown | | json | Machine-readable array of site results |

Credits

This tool is built on top of afdocs — the Agent-Friendly Documentation audit engine and spec created by Dachary Carey, Developer Experience Engineer at MongoDB.

Dachary created the Agent-Friendly Documentation Spec to define what makes documentation consumable by AI agents, and built afdocs as the reference implementation. Without that foundational work, this tool wouldn't exist.

• Website: dacharycarey.com
• GitHub: @dacharyc
• npm: npmjs.com/~dacharyc

Requirements

• Node.js >= 22
• macOS / Linux / Windows

Development

git clone https://github.com/Chris-ganta/llms-agent-score
cd llms-agent-score
npm install
npm run build       # compile TypeScript → dist/
npm run dev         # watch mode
npm test            # run tests with vitest

Run locally:

node bin/agent-score.mjs score https://docs.example.com