@crossplatformai/deduper

AST-based duplicate detection for TypeScript and CSS projects. Detects duplicate code patterns using TypeScript Abstract Syntax Tree analysis plus first-class CSS normalization.

Overview

The deduper scans your codebase for duplicate code patterns such as types, functions, classes, components, and CSS blocks. It uses TypeScript's AST parser for TypeScript analysis and CSS normalization for stylesheet analysis, detecting duplicates based on structure rather than just raw text matching.

Quick Start

# Run without installing
npx @crossplatformai/deduper

# Or install as a dev dependency
npm install --save-dev @crossplatformai/deduper

Add to your package.json scripts:

{
  "scripts": {
    "check-duplicates": "deduper"
  }
}

Then run:

npm run check-duplicates

Available Detectors

The package includes 14 specialized detectors:

| Detector | Description | | ------------------- | ------------------------------------------- | | types | TypeScript type aliases and interfaces | | css | Whole stylesheet and top-level CSS at-rules | | functions | Function declarations and expressions | | constants | Constant variable declarations | | classes | ES6 class declarations | | abstract-classes | Abstract class declarations | | components | React/JSX components | | enums | TypeScript enum declarations | | hooks | React hooks (functions starting with use) | | type-guards | TypeScript type guard functions | | zod-schemas | Zod schema definitions | | routes | Route definitions (API/web routes) | | context-providers | React context provider components | | error-classes | Custom error class declarations |

CLI Usage

Run All Detectors

By default, running without arguments checks for all duplicate patterns:

deduper

Or explicitly:

deduper --all

Run Specific Detector

Check for duplicates of a specific type:

# Check for duplicate type definitions
deduper types

# Check for duplicate functions
deduper functions

# Check for duplicate React components
deduper components

# Check for duplicate CSS
deduper css

# Check for duplicate React hooks
deduper hooks

CLI Options

deduper --help

| Option | Description | | --------------------- | ------------------------------------- | | --all | Run all available detectors (default) | | --config, -c <path> | Path to configuration file | | --no-config | Disable configuration file loading | | --show-allowed | Show allowed duplicates in output | | --suggest | Show copy-paste config snippets | | --help, -h | Show help message |

Configuration files can be named deduper.config.ts, deduper.config.mjs, or deduper.config.js.

Output Format

When duplicates are found, the output shows:

Reported Duplicates:

  Group 1
    Name: User
    Kind: interface
    Files (2):
      packages/auth/src/types.ts (@repo/auth)
      packages/api/src/types.ts (@repo/api)

Found 1 duplicate group(s) to address

When no duplicates are found:

No duplicate code to report

Config Suggestions

Use --suggest to get copy-paste config snippets:

deduper types --suggest

This adds a ready-to-use config snippet for each duplicate:

{
  name: "User",
  kind: "interface",
  files: [
    "packages/auth/src/types.ts",
    "packages/api/src/types.ts"
  ],
  reason: "TODO: explain why this duplication is intentional"
}

Exit Codes

• 0: No duplicates found
• 1: Duplicates found or error occurred

This makes the tool useful in CI/CD pipelines:

# Fail the build if duplicates are detected
deduper || exit 1

Common Use Cases

Pre-commit Hook

Add to your pre-commit workflow to catch duplicates before they're committed:

{
  "scripts": {
    "check-duplicates": "deduper"
  }
}

CI/CD Pipeline

Run in your CI pipeline to enforce code quality:

- name: Check for code duplicates
  run: npx @crossplatformai/deduper

Code Review

Use specific detectors during code review to focus on particular patterns:

# When reviewing API changes
deduper types

# When reviewing component changes
deduper components

# When reviewing error handling
deduper error-classes

Refactoring Analysis

Identify candidates for consolidation:

# Find all duplicate utilities
deduper functions

# Find duplicate constants that could be centralized
deduper constants

Library Usage

Import and use the deduper programmatically:

import {
  detect,
  loadConfig,
  detectDuplicates,
  TypeDetectorPlugin,
  FunctionDetectorPlugin,
} from '@crossplatformai/deduper';

// Simple API - detect specific kind
const results = await detect('types', {
  threshold: 2,
});

// Advanced API - full control
const config = await loadConfig();
const result = await detectDuplicates(
  files, // string[] of file paths
  config,
  [new TypeDetectorPlugin(), new FunctionDetectorPlugin()],
);

See the TypeScript types for full API documentation.

Configuration

Deduper looks for deduper.config.ts, then deduper.config.mjs, then deduper.config.js in your project root (and parent directories).

File Discovery and Ignores

Important: There are no built-in ignore patterns. You control which files are excluded from detection via global.ignore in your configuration file.

When no configuration is found, all .ts, .tsx, and .css files are scanned, including files in node_modules, dist, .next, etc. The CSS detector still applies its own built-in skip-directory filter for common generated and dependency folders.

To exclude common build and dependency directories, add them to your config:

/** @type {import('@crossplatformai/deduper').DeduperConfig} */
export default {
  global: {
    ignore: [
      '**/node_modules/**',
      '**/dist/**',
      '**/.turbo/**',
      '**/.next/**',
      '**/out/**',
    ],
  },
};

Use --no-config to run without any configuration (and thus without any ignore patterns):

# Scan everything, including node_modules and build directories
deduper --no-config

Basic Configuration

Create deduper.config.mjs in your project root:

/** @type {import('@crossplatformai/deduper').DeduperConfig} */
export default {
  allowRules: [
    {
      name: 'AppType',
      kind: 'type',
      files: ['apps/api-origin/src/index.ts', 'apps/api-edge/src/index.ts'],
      reason: 'Each API exports its own Hono instance type for RPC clients',
    },
  ],
};

TypeScript Configuration (Development Only)

If you prefer CommonJS, you can use deduper.config.js with module.exports = {}. If you want ESM syntax in plain JavaScript, use deduper.config.mjs. If you are developing within a TypeScript project that has tsx or ts-node installed, you can use deduper.config.ts:

import type { DeduperConfig } from '@crossplatformai/deduper';

export default {
  allowRules: [
    {
      name: 'AppType',
      kind: 'type',
      files: ['apps/api-origin/src/index.ts', 'apps/api-edge/src/index.ts'],
      reason: 'Each API exports its own Hono instance type for RPC clients',
    },
  ],
} satisfies DeduperConfig;

Note: The published CLI requires a JavaScript config file unless your environment has TypeScript execution capabilities installed.

Architecture

Plugin System

Each detector implements the DetectorPlugin interface:

interface DetectorPlugin {
  name: string;
  detect(files: string[], config: DetectionConfig): Promise<DuplicateGroup[]>;
}

Detection Process

1. File Scanning: Glob all .ts, .tsx, and .css files (respecting global.ignore patterns from config)
2. AST Parsing: Parse TypeScript files using the TypeScript compiler API
3. Pattern Extraction: Each detector extracts its specific pattern type
4. Signature Normalization: Generate canonical signatures for comparison
5. Duplicate Grouping: Group items with identical names and signatures
6. Package Resolution: Extract package names from file paths
7. Result Formatting: Format and display duplicate groups

Adding New Detectors

To add a new detector:

1. Create a new file in src/detectors/
2. Implement the DetectorPlugin interface
3. Export from src/detectors/index.ts
4. Register in src/detect.ts DETECTORS map

Development

Building

pnpm build

This builds both the library (dist/index.js) and CLI (dist/cli.js) with TypeScript declarations.

Running Tests

pnpm test

Type Checking

pnpm check-types

Linting

pnpm lint

Testing

This package uses a co-located testing pattern where test files are placed next to the source files they test:

• Co-located tests (src/**/*.test.ts): Tests placed next to source code, testing individual detectors and utilities

Run all tests:

pnpm test

The Vitest configuration discovers all .test.ts files throughout the src directory.

Publishing

This package is published to npm as @crossplatformai/deduper.

# Build first
pnpm build

# Preview the release recommendation without mutating npm
pnpm release --dry-run

# Run the guided release flow
pnpm release

The prepack script ensures the package is built before publishing, and direct pnpm publish is intentionally blocked in favor of the guided pnpm release flow.

Technical Details

• AST Analysis: Uses TypeScript Compiler API for accurate parsing
• Monorepo-Aware: Automatically detects package boundaries
• Performance: Processes files in parallel where possible
• Error Handling: Gracefully skips unparseable files
• Output Formatting: Uses picocolors for readable terminal output
• Build Tool: Uses tsup for fast bundling with ESM output

Limitations

• Only scans .ts, .tsx, and .css files
• TypeScript detectors require valid TypeScript syntax
• Signature comparison is structure-based (may not catch semantically identical but syntactically different code)
• Does not analyze runtime behavior or values
• Config loading requires native ESM support (Node 20+)

Requirements

• Node.js 20 or higher
• TypeScript projects (for AST parsing)

See Configuration Guide for detailed documentation.