Why

AI coding agents read your CLAUDE.md, AGENTS.md, or .cursorrules, but nothing checks whether your ESLint config actually enforces the same rules. Drift accumulates: the instruction file says "use camelCase" but ESLint never checks it, or ESLint enforces rules the instruction file never mentioned.

RuleProbe closes that gap. It reads your instruction file, extracts machine-verifiable rules, and bridges them to your tooling in three ways:

Translate. Generate an ESLint config from your instruction file. ruleprobe lint-config CLAUDE.md produces a flat or legacy config you can drop into your project immediately.

Detect drift. Compare your instruction file against an existing ESLint config. ruleprobe drift CLAUDE.md .eslintrc.json reports rules present in one but missing from the other, severity mismatches, and config argument differences.

Extract. Pull a rules section out of an ESLint config for pasting into an instruction file. ruleprobe extract .eslintrc.json converts ESLint rules back into human-readable instruction prose.

Quick Start

npm install -g ruleprobe

Or run it directly:

npx ruleprobe --help

Translate an instruction file to ESLint config:

ruleprobe lint-config CLAUDE.md
ruleprobe lint-config AGENTS.md --format legacy --output .eslintrc.json

Detect drift between instructions and ESLint:

ruleprobe drift CLAUDE.md .eslintrc.json
ruleprobe drift CLAUDE.md .eslintrc.json --format markdown

Extract rules from an ESLint config:

ruleprobe extract .eslintrc.json
ruleprobe extract .eslintrc.json --output rules-section.md

Parse an instruction file to see what rules RuleProbe can extract:

ruleprobe parse CLAUDE.md
ruleprobe parse AGENTS.md --show-unparseable

Verify agent output against extracted rules (legacy mode). Valid formats: text, json, markdown, rdjson, summary, detailed, ci.

ruleprobe verify CLAUDE.md ./agent-output --format text

What It Does

Translate. Reads 7 instruction file formats (CLAUDE.md, AGENTS.md, .cursorrules, copilot-instructions.md, GEMINI.md, .windsurfrules, .rules) and emits an ESLint config. Flat config is the default; pass --format legacy for .eslintrc.json output. Each extractable rule maps to an ESLint rule with appropriate severity and options. Rules that have no ESLint equivalent appear as comments in the output so you know what wasn't covered.

Detect drift. Compares parsed rules against an existing ESLint config. Reports rules in the instruction file but missing from ESLint (you're not enforcing what you said), rules in ESLint but not in the instructions (you're enforcing what you never stated), severity mismatches, and argument differences. Use --format markdown for PR-ready output.

Extract. Parses an ESLint config and generates a markdown rules section you can paste into an instruction file. Stylistic rules (semicolons, quotes) are reported but skipped from the output by default since they don't carry meaningful instruction content.

Parse. Extracts machine-verifiable rules from instruction files with qualifiers (always, prefer, when-possible, avoid-unless, try-to, never) and section context. Subjective lines like "write clean code" are reported as unparseable.

Verify. Runs extracted rules against a directory of code. Deterministic by default, optional semantic analysis available. The original mode, still supported but no longer the primary focus.

Analyze. Discovers all instruction files in a project, parses each, and cross-references them for conflicts and redundancies. Pass --semantic for structural pattern analysis.

Configuration

RuleProbe auto-discovers a config file in the working directory (or any parent). You can also pass --config <path> explicitly. Supported file names, in priority order:

• ruleprobe.config.ts
• ruleprobe.config.js
• ruleprobe.config.json
• .ruleproberc.json

A config file lets you add custom rules, override extracted rules, or exclude rules entirely:

// ruleprobe.config.ts
import { defineConfig } from 'ruleprobe';

export default defineConfig({
  // Add rules that the parser can't extract from your instruction file
  rules: [
    {
      id: 'custom-no-lodash',
      category: 'import-pattern',
      description: 'Ban lodash imports',
      verifier: 'regex',
      pattern: { type: 'banned-import', target: '*.ts', expected: 'lodash', scope: 'file' },
    },
  ],

  // Change severity or expected values on extracted rules
  overrides: [
    { ruleId: 'naming-camelcase', severity: 'warning' },
    { ruleId: 'structure-max-file-length', expected: '500' },
  ],

  // Remove rules you don't want checked
  exclude: ['forbidden-no-console-log'],
});

defineConfig() is a no-op passthrough that provides type checking in TypeScript configs. JSON configs work without it.

Custom rules use the same verifier types (ast, regex, filesystem) and pattern types as extracted rules.

CLI Reference

Six commands: parse, verify, analyze, lint-config, drift, extract.

ruleprobe parse CLAUDE.md --show-unparseable
ruleprobe verify AGENTS.md ./src --format detailed --threshold 0.9
ruleprobe lint-config CLAUDE.md --format flat --output eslint.config.js
ruleprobe drift CLAUDE.md .eslintrc.json --format markdown
ruleprobe extract .eslintrc.json --output rules-section.md
ruleprobe analyze ./my-project --format json

Full command reference with all options: docs/cli-reference.md

GitHub Action

Drop this into .github/workflows/ruleprobe.yml:

name: RuleProbe Drift
on: [pull_request]
jobs:
  drift-check:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v4
      - uses: moonrunnerkc/ruleprobe@v4
        with:
          instruction-file: CLAUDE.md
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

No API keys needed, deterministic results, runs in seconds. The action only runs drift detection when instruction files or ESLint config files change in the PR. @v4 tracks the latest v4.x release; pin to a specific tag (e.g., @v4.5.0) for reproducible builds.

- uses: moonrunnerkc/ruleprobe@v4
  with:
    mode: drift
    instruction-file: CLAUDE.md
    eslint-file: .eslintrc.json
    regenerate-on-drift: "false"
    comment-on-pr: "true"
    fail-on-drift: "false"

| Input | Default | Description | |-------|---------|-------------| | mode | drift | Run mode: drift (default) or verify (legacy) | | instruction-file | (required) | Path to instruction file | | eslint-file | (auto-detected) | Path to ESLint config file | | regenerate-on-drift | false | Open a follow-up PR with regenerated config when drift is detected | | comment-on-pr | true | Post drift results as a PR comment | | fail-on-drift | false | Fail the action if drift is detected |

Drift mode outputs: drift-count, has-drift.

Programmatic API

Core pipeline functions are exported for programmatic use:

import { parseInstructionFile, verifyOutput, generateReport, formatReport } from 'ruleprobe';

const ruleSet = parseInstructionFile('CLAUDE.md');
const results = await verifyOutput(ruleSet, './agent-output');
const report = generateReport(
  { agent: 'claude-code', model: 'opus-4', taskTemplateId: 'manual',
    outputDir: './agent-output', timestamp: new Date().toISOString(), durationSeconds: null },
  ruleSet,
  results,
);
console.log(formatReport(report, 'summary'));

Full API reference: docs/api-reference.md

How It Works

Instruction File --> Rule Parser --> RuleSet --+
                                               +--> Verifier --> Adherence Report
                        Agent Output ---------+

Instruction File --> Rule Parser --> RuleSet --> Mapper --> ESLint Config

Instruction File --> Rule Parser --> RuleSet --+
ESLint Config     --> Parser     --> Parsed --+--> Drift Report

ESLint Config     --> Extractor --> Markdown Rules Section

The parser reads your instruction file and identifies lines that map to deterministic checks. Each rule gets a category, a verifier type, a pattern, and a qualifier (how strictly the instruction is worded). Three verifier engines handle different rule types:

| Engine | What it checks | |--------|---------------| | AST (ts-morph) | Code structure, naming, type safety, imports for TypeScript/JavaScript | | Filesystem | File existence, naming conventions, directory structure | | Regex | Content patterns, forbidden strings |

The mapper translates extractable rules into ESLint rule entries. The drift detector compares parsed rules against an existing ESLint config. The extractor reverses the mapping: ESLint rules become human-readable instruction prose.

Supported Rule Types

34 mappable matchers across 7 categories produce ESLint config entries:

| Category | Count | Verifier | Examples | |----------|------:|----------|----------| | naming | 5 | AST, Filesystem | camelCase variables, PascalCase types, kebab-case files, UPPER_CASE constants | | forbidden-pattern | 5 | AST, Regex | no any, no console.log, no var, max line length, no TODO comments | | structure | 4 | AST, Filesystem | named exports, JSDoc required, max file length, no unused exports | | import-pattern | 4 | AST | no path aliases, no deep relative imports, no namespace imports, no wildcard exports | | error-handling | 2 | AST | no empty catch, throw Error only | | type-safety | 5 | AST, Regex | no enums, no type assertions, no non-null assertions, no implicit any, no ts directives | | code-style | 9 | AST, Regex | prefer const, no else after return, no nested ternary, no magic numbers, semicolons, quotes, max function length, max params, no TODO comments |

Rules that can't map to ESLint (test file requirements, project config, git conventions) are reported as unmappable so you can enforce them through other tooling.

Full matcher details: docs/matchers.md

Security

RuleProbe never executes scanned code, never makes network calls (unless you opt in with --llm-extract, --rubric-decompose, or --semantic), and never modifies files in the scanned directory. User-supplied paths are resolved and bounded to the working directory; symlinks outside the project are skipped unless you pass --allow-symlinks. All dependencies are pinned to exact versions.

When --semantic is enabled, all analysis runs locally. The only network calls are to the Anthropic API for LLM judgment when vector similarity is ambiguous (using your own ANTHROPIC_API_KEY). Only numeric AST vectors, opaque sub-tree hashes, boolean flags, and rule text are sent to the LLM. No source code, variable names, comments, import paths, or file paths leave the machine. See SECURITY.md for the full model.

Limitations

• TypeScript gets the deepest coverage. ts-morph gives full AST analysis for TypeScript and JavaScript across all categories. Other languages get regex-based checks only.
• Subjective rules stay subjective. "Write clean code" has no deterministic check. --rubric-decompose uses an LLM to break subjective instructions into weighted concrete checks, tagged with confidence: 'low'. Requires OPENAI_API_KEY.
• Not all rules map to ESLint. Test file requirements, project config conventions, git workflow rules, and preference pairs don't have ESLint equivalents. These are reported as unmappable so you can enforce them through other tooling.
• Monorepo support is limited. Drift detection and lint-config scan from the repository root and use the first ESLint config found. Monorepos with per-package instruction files or configs need to specify paths explicitly.

What's New in v4.5.0

v4.5.0 pivots RuleProbe from "verify adherence" to "translate instruction files into ESLint configs." The core value proposition is now: translate, detect drift, extract.

Key changes:

• New commands: lint-config (translate instructions → ESLint config), drift (compare instructions vs an existing ESLint config), extract (ESLint config → instruction prose).
• Removed: compare, tasks, task, run, the runner module exports (buildAgentConfig, invokeAgent, watchForCompletion, countCodeFiles).
• Deprecated: verify still works, but the primary workflow is now lint-config, drift, and extract.
• Matcher audit: 34 ESLint-mappable matchers remain across 7 categories (naming, forbidden-pattern, structure, import-pattern, error-handling, type-safety, code-style). 67 unmappable matchers and the test-requirement, dependency, preference, file-structure, tooling, testing, and workflow categories were removed.

Contributing

git clone https://github.com/moonrunnerkc/ruleprobe.git
cd ruleprobe && npm install
npm test

Issues and pull requests welcome at github.com/moonrunnerkc/ruleprobe.

License

MIT