@devrelkit/docs-check

PR-driven documentation gap detection. Given a code pull request and a documentation source, identify which docs files need updating and why.

The package is the engine. It does not post anywhere — it returns a typed result. Wrap it in a GitHub App, a CLI, or a CI step depending on what you're building.

Install

npm install @devrelkit/docs-check @anthropic-ai/sdk

Quick start

import Anthropic from '@anthropic-ai/sdk'
import { checkDocs, githubDiff, githubDocs, renderMarkdownComment } from '@devrelkit/docs-check'

const result = await checkDocs({
  diff: await githubDiff({
    owner: 'anza-xyz',
    repo: 'kit',
    prNumber: 123,
    token: process.env.GITHUB_TOKEN,
  }),
  docs: githubDocs({
    owner: 'solana-foundation',
    repo: 'solana-com',
    token: process.env.DOCS_GITHUB_TOKEN,
    pathPrefix: 'content/docs',
  }),
  mapping: [
    {
      source: 'packages/kit/src/rpc/**',
      docs: 'content/docs/clients/kit/rpc/**',
      description: 'Kit RPC client and request shapes',
    },
  ],
  anthropic: new Anthropic(),
})

if (result.needsDocs) {
  console.log(renderMarkdownComment(result))
}

What it does

1. Apply mapping rules — for each rule, check if any changed source file matches the rule's source glob. If no rule activates, return early without calling the model.
2. Resolve docs globs — for active rules, find docs files matching the rule's docs glob.
3. Fetch matched docs — read current contents from the docs source.
4. Ask Claude — given the diff, the matched docs, and the mapping context, decide which docs are stale and why.
5. Return a typed result — caller decides how to render or post.

What it does not do

• Post comments, open issues, or open PRs — that's the consumer's job.
• Handle webhooks or GitHub App auth.
• Dedupe issues across PR pushes or manage issue lifecycle.
• Know anything about your specific repos.

This split keeps the engine reusable across orgs and surfaces.

API

checkDocs(params)

checkDocs({
  diff: Diff,
  docs: DocsSource,
  mapping: MappingRule[],
  anthropic: Anthropic,        // BYO @anthropic-ai/sdk client
  options?: CheckDocsOptions,
}): Promise<DocsCheckResult>

Options:

• suggest: 'analysis' | 'patch' | 'replace' (default 'analysis'):
  • 'analysis' — findings only, no edits.
  • 'patch' — findings + unified-diff suggestedPatch. Apply with applyPatch. Brittle to whitespace/context drift.
  • 'replace' — findings + find/replace suggestedEdits. Apply with applyEdits. More reliable than unified diffs; recommended over 'patch'.
• model: string (default 'claude-opus-4-7') — any Claude model ID.
• effort: 'low' | 'medium' | 'high' | 'xhigh' | 'max' (default 'high') — passed to output_config.effort.
• maxDocsFiles: number (default 25) — caps how many docs files are fetched and sent to the model. Tighten your mapping globs if you regularly hit this.
• systemPromptExtra: string — appended to the engine's system prompt for repo-specific rules ("treat anything under internal/ as not user-facing").
• maxApplyRetries: number (default 2) — for 'patch' and 'replace', the engine validates each proposed edit against the current file content. If edits don't apply, it asks the model to correct them (one model call per retry). Set 0 to disable. Each finding ends up with applied: true | false so consumers can route auto-applicable findings to a PR and the rest to manual review.

MappingRule

{
  source: string | string[]   // globs against changed file paths
  docs: string | string[]     // globs against the docs source's file list
  description?: string        // shown to the model as rule context
}

Globs are matched with picomatch — standard ** / * semantics.

DocsSource

Two methods. The package ships githubDocs(...) as the common implementation; you can also pass your own (local FS, S3, mock for tests, monorepo of docs).

interface DocsSource {
  listFiles(): Promise<string[]>
  getFile(path: string): Promise<string | null>
}

DocsCheckResult

{
  needsDocs: boolean
  findings: Array<{
    docsFile: string
    summary: string             // why this doc is stale
    severity: 'high' | 'low'
    suggestedPatch?: string     // unified diff, only when suggest: 'patch'
    suggestedEdits?: Array<{    // find/replace, only when suggest: 'replace'
      find: string              // must appear exactly once in the file
      replace: string
    }>
    applied?: boolean           // true if the patch/edits applied cleanly,
                                // false if they failed even after retries.
                                // undefined when suggest is 'analysis'.
  }>
  changedFiles: string[]
  matchedSources: string[]
  matchedDocs: string[]
  skippedReason?: 'no-changed-files' | 'no-mapping-match' | 'no-matched-docs'
  reasoning?: string
  usage?: { inputTokens, outputTokens, cacheReadTokens, cacheCreationTokens }
}

Apply utilities

import { applyEdits, applyPatch } from '@devrelkit/docs-check'

// For suggest: 'replace' findings
const r1 = applyEdits(currentContent, finding.suggestedEdits)
// → { ok: true, newContent: '...' } or { ok: false, reason: 'find-not-found' }

// For suggest: 'patch' findings
const r2 = applyPatch(currentContent, finding.suggestedPatch)
// → { ok: true, newContent: '...' } or { ok: false, reason: 'patch-did-not-apply' }

applyEdits requires each find string to occur exactly once in the (current) content; zero or multiple matches are both rejected. applyPatch is a strict unified-diff applier — context lines must match the file exactly. Both are pure functions; neither writes to disk.

Renderers

• renderMarkdownComment(result, { docsRepoUrl?, title?, includeReasoning? }) — markdown for a PR comment.
• renderIssueBody(result, { sourcePrUrl, sourceRepo, docsRepoUrl? }) — markdown for an issue in your docs repo, with a hidden marker for find-or-create dedup.

GitHub helpers

• githubDiff({ owner, repo, prNumber, token }) — fetches PR metadata and the file list with patches.
• githubDocs({ owner, repo, branch?, token?, pathPrefix? }) — implements DocsSource against a GitHub repo. pathPrefix constrains listFiles to a subtree (recommended for large docs repos).

Design notes

• BYO Anthropic client. The package never instantiates Anthropic. Pass your own so you control timeouts, retries, observability, and which API key gets billed.
• No state. No DB, no caching across calls, no dedup logic. Wrap with caching if you want it.
• Pluggable sources. Diff and DocsSource are plain interfaces — works with GitLab, local FS, monorepos, or test fixtures.
• Cheap exit. When no mapping rule matches the changed files, the package returns immediately without calling the model.

License

MIT