crates-llms-txt

This library provides a standard interface to fetch and parse Rust crate documentation and session data for use with LLMs (Large Language Models).

Features

• Generate metadata for llms.txt、 llms-full.txt files
• Parse Rust crate documentation into standardized formats
• Support both online (docs.rs) and local crate documentation
• Enable custom feature selection for local documentation generation
• Cross-platform support with prebuilt binaries

Installation

npm install crates-llms-txt

Usage Examples

Basic Usage - Fetch by Crate Name

import { fromCrateName } from 'crates-llms-txt'

async function main() {
  // Fetch latest version
  const config = await fromCrateName('clap')

  // Fetch specific version
  const specificConfig = await fromCrateName('clap', '4.5.39')

  if (config) {
    console.log(`Fetched docs for ${config.libName} v${config.version}`)
    console.log(`Found ${config.sessions.length} documentation sections`)
  }
}

main()

Advanced Usage - Local Documentation Generation

import { fromLocal, fromLocalWithFeatures } from 'crates-llms-txt'

// Generate docs with all features
const allFeaturesConfig = fromLocal('./Cargo.toml', 'stable')

// Generate docs with specific features
const customConfig = fromLocalWithFeatures(
  './Cargo.toml',
  true, // no default features
  ['async', 'serde'], // specific features
  'nightly', // toolchain
)

API Reference

Online Documentation Functions

fromCrateName(libName: string, version?: string): Promise<LLMsConfig | null>

Fetches Rust crate documentation from docs.rs by crate name and version.

• libName: string: The name of the crate as it appears on crates.io (e.g., "clap", "serde", "tokio")
• version?: string: Optional version string. If not provided, the latest version will be fetched
• Returns: Promise<LLMsConfig | null> - Documentation configuration or null if failed

fromUrl(url: string): Promise<LLMsConfig | null>

Fetches documentation from a direct URL to the JSON documentation.

• url: string: Direct URL to the crate's JSON documentation (e.g., "https://docs.rs/crate/clap/latest/json")
• Returns: Promise<LLMsConfig | null> - Documentation configuration or null if failed

fromOnline(params: LLMsConfigByCrate | LLMsConfigByUrl): Promise<LLMsConfig | null>

Unified function for fetching documentation from online sources.

• params: Either { libName: string, version?: string } or { url: string }
• Returns: Promise<LLMsConfig | null> - Documentation configuration or null if failed

Local Documentation Functions

fromLocal(manifestPath: string, toolchain?: string): LLMsConfig | null

Generates documentation for a local crate with all features enabled.

• manifestPath: string: Path to the Cargo.toml file
• toolchain?: string: Optional Rust toolchain (e.g., "stable", "nightly")
• Returns: LLMsConfig | null - Documentation configuration or null if failed

fromLocalWithFeatures(manifestPath: string, noDefaultFeatures: boolean, features?: string[], toolchain?: string): LLMsConfig | null

Generates documentation with fine-grained feature control.

• manifestPath: string: Path to the Cargo.toml file
• noDefaultFeatures: boolean: Whether to disable default features
• features?: string[]: Optional array of features to enable
• toolchain?: string: Optional Rust toolchain
• Returns: LLMsConfig | null - Documentation configuration or null if failed

fromLocalByRustdoc(params: LLMsConfigRustdocByAllFeatures | LLMsConfigRustdocByFeatures): LLMsConfig | null

Unified function for local documentation generation with flexible configuration.

• params: Either all-features config or specific-features config
• Returns: LLMsConfig | null - Documentation configuration or null if failed

TypeScript Types

interface SessionItem {
  title: string
  description: string
  link: string
}

interface FullSessionItem {
  content: string
  link: string
}

interface LLMsConfig {
  libName: string
  version: string
  sessions: SessionItem[]
  fullSessions: FullSessionItem[]
}

interface LLMsConfigByCrate {
  libName: string
  version?: string
}

interface LLMsConfigByUrl {
  url: string
}

interface LLMsConfigRustdocByAllFeatures {
  toolchain?: string
  manifestPath: string
}

interface LLMsConfigRustdocByFeatures {
  toolchain?: string
  manifestPath: string
  noDefaultFeatures: boolean
  features?: string[]
}

Supported Architectures

The crates-llms-txt package provides prebuilt binaries for the following target architectures:

| Target Triple | | ------------------------------- | | x86_64-apple-darwin | | aarch64-apple-darwin | | x86_64-pc-windows-msvc | | aarch64-pc-windows-msvc | | i686-pc-windows-msvc | | x86_64-unknown-linux-gnu | | x86_64-unknown-linux-musl | | aarch64-unknown-linux-gnu | | aarch64-unknown-linux-musl | | armv7-unknown-linux-gnueabihf |