Refactor Check Tool

A tool to validate that PR changes conform to specified refactoring patterns, enabling safer automated code transformations.

Purpose

When performing automated refactorings (like adding type annotations across many files), you want to ensure that ONLY the intended changes were made. This tool:

1. Checks that all changes in a PR match explicitly defined transformation patterns
2. Flags files with non-conforming changes for manual review
3. Makes PR review faster and more confident for large automated refactorings

Installation

npm install -g refactor-check

Or run directly with npx:

npx refactor-check <pr-number> [pattern-file]

Usage

refactor-check <pr-number> [pattern-file]

Examples:

# Use pattern from PR description (```refactor-check block)
refactor-check 1821

# Use local pattern file
refactor-check 1821 example-patterns/add-props-types.yaml

Pattern From PR Description

If you don't specify a pattern file, the tool will fetch the pattern from the PR description. Add a refactor-check code block to your PR description:

## Refactor Pattern

```refactor-check
allowed_patterns:
  - change: "Add type annotation"
    before: "function ${Name}({ ${Params} })"
    after: "function ${Name}({ ${Params} }: ${Type})"

excluded_files:
  - path/to/file.tsx
```

Pattern File Format

Pattern files are written in YAML and specify exactly what code transformations are allowed.

Basic Structure

name: "Refactoring Name"

allowed_patterns:
  - change: "Description of what this pattern does"
    before: "code pattern before"
    after: "code pattern after"

excluded_files:
  path/to/file.tsx: "Reason this file has other changes"

Pattern Syntax

Patterns use holes to match code transformations:

• ${Name} - Matches any text that doesn't contain unbalanced delimiters ()[]{}
• Holes with the same name must match the same text in before/after patterns
• Holes can match inside strings: '${Path}/firebaseutil.js' matches '../util/firebaseutil.js'
• Note: All patterns ignore whitespace differences by default, making them robust to code formatting changes.

Pattern Combination

The tool automatically combines "add" and "remove" patterns:

• If you have a pattern that removes X (before: X, after: "")
• And a pattern that adds Y (before: "", after: Y)
• They automatically combine to match hunks that change X to Y

This means you don't need explicit "change" patterns - just define what can be added and removed.

Path-Agnostic Patterns

Use ${Path} to match any path prefix, making patterns work regardless of import style:

- change: "Remove import from firebaseutil (any path)"
  before: "import { ${Items} } from '${Path}/firebaseutil.js';"
  after: ""

This matches:

• import { foo } from 'util/firebaseutil.js';
• import { foo } from '../util/firebaseutil.js';
• import { foo } from '../../util/firebaseutil.js';

Example Patterns

Changing Import Paths

- change: "Change import from one path to another"
  before: "import { ${Items} } from '${FromPath}';"
  after: "import { ${Items} } from '${ToPath}';"

This matches:

// Before
import { foo, bar } from '../util/firebaseutil.js';

// After
import { foo, bar } from '../util/keyutil.js';

The ${Items} must match exactly in before/after (same items imported).

Adding Generic Types to Hooks

- change: "Add generic type to hook"
  before: "${Hook}(${Args})"
  after: "${Hook}<${Type}>(${Args})"

This matches:

// Before
const [items, setItems] = useState([])

// After
const [items, setItems] = useState<string[]>([])

Adding Imports

- change: "Add import statement"
  before: ""
  after: "import { ${Items} } from '${Path}';"

This matches any new import being added.

Output

The tool will:

✅ Pass - If all changes match the allowed patterns

✅ All changes conform to the allowed patterns!
✅ PR #1821 is ready for review.

❌ Fail - If some files have non-conforming changes

❌ Found 3 file(s) with non-conforming changes:

  client/feature/topic/reading-list.tsx
    Contains changes not matching allowed patterns
    (2 hunk(s) don't match any pattern)

📝 Suggested additions to excluded_files:
  client/feature/topic/reading-list.tsx: "Contains changes not matching allowed patterns"

Excluded Files

Files listed in excluded_files are skipped during pattern checking. Use this for files that legitimately need other changes beyond the refactoring.

Exclusion Syntax

excluded_files:
  # Exclude specific file
  client/some/file.tsx: "Also fixes a bug in validation logic"

  # Exclude entire folder (note trailing /)
  server/adapter/: "New adapter implementation files"

  # Exclude pattern with wildcards
  **/*.test.ts: "Test files have additional changes"
  server/modules/*.ts: "All module files need review"

Supported patterns:

• Exact path: path/to/file.ts - Matches only that specific file
• Folder: path/to/folder/ - Matches all files in that folder (note trailing /)
• Wildcards:
  • * matches any characters except / (single folder level)
  • ** matches any characters including / (multiple folder levels)
  • ? matches any single character

Examples:

• server/adapter/ - Excludes server/adapter/auth/types.ts, server/adapter/database/firebase.ts, etc.
• **/*.test.ts - Excludes all test files in any folder
• server/modules/*.ts - Excludes all .ts files directly in server/modules/

How It Works

1. Fetches PR diff using gh pr diff
2. Parses hunks - each contiguous change in the diff
3. Pattern matching - converts pattern templates to regex, matches against each hunk
4. AST-aware - uses template variables to match code structure, not just text
5. Reports - lists files with non-matching changes

Example: PR #1821

PR #1821 adds TypeScript prop types to React components. The pattern file specifies:

• Adding type definitions
• Adding type annotations to function parameters
• Adding generic types to hooks
• Adding imports for types

Result: 34/37 files matched the patterns perfectly. 3 files were flagged because they had additional changes (modifying existing type definitions), which correctly required manual review.

Tips

• Start specific - Write patterns for the exact transformations you're making
• Test on real PRs - Run the tool on your PR to see what doesn't match
• Iterate patterns - Add patterns for legitimate variations you discover
• Use excluded_files - For files that genuinely need other changes
• Keep it simple - Patterns should be easy for humans to read and verify

Files

• index.js - Main tool implementation
• hole-matcher.js - Hole-based pattern matching engine
• example-patterns/add-props-types.yaml - Example pattern file for adding React prop types
• example-patterns/auth-adapter-refactor.yaml - Example pattern file for auth adapter refactoring
• README.md - This file

Requirements

• Node.js 20+
• gh CLI (GitHub CLI) installed and authenticated
• Access to the repository containing the PR

License

Apache-2.0