conform

Automated validation for Claude Code plugins and shared .claude configurations. Structural checks, integration tests, response logging, and linting — with a terminal UI or JSON output for CI.

Install

npm install -g @iwritec0de/conform
# or
pnpm add -g @iwritec0de/conform
# or
yarn global add @iwritec0de/conform

Or run locally from the repo:

pnpm install && pnpm build
./conform ./my-plugin

What It Validates

Conform works with three target types:

| Target | Detection | What's Validated | |--------|-----------|------------------| | Plugin directory | Contains .claude-plugin/, skills/, commands/, agents/, or hooks/ | Manifest + all components | | .claude project directory | .claude/ folder with commands, skills, agents, or hooks inside | Components only (no manifest required) | | Directory of plugins | Parent folder containing multiple plugin dirs | Each child plugin independently |

conform ./my-plugin                    # Plugin with manifest
conform structural .claude             # Validate .claude project directory
conform ./plugins/                     # Scan all plugins in a directory
conform ./plugins/ --skip drafts,wip   # Skip specific subdirectories

Modes

| Mode | Description | |------|-------------| | structural | Static validation — free, instant, no API calls | | integration | Integration tests against Claude API (claude -p) | | response-log | Run user-defined prompts and capture full responses | | lint | Fast structural linter with error/warning output | | all | Structural + integration (default) |

conform ./my-plugin                              # Default: all (structural + integration)
conform structural ./my-plugin                   # Structural only
conform integration ./my-plugin                  # Integration only
conform response-log ./my-plugin                 # Run test-prompts.json
conform lint ./my-plugin                         # Quick lint

Authentication

Integration and response-log modes need access to the Claude API. Two auth modes are supported:

| Mode | How It Works | When to Use | |------|-------------|-------------| | oauth (default) | Uses your native claude CLI login session | Local development, interactive use | | api | Uses ANTHROPIC_API_KEY environment variable | CI pipelines, headless environments |

Auth is auto-detected: if ANTHROPIC_API_KEY is set, api mode is used. Otherwise falls back to oauth.

# Native Claude CLI (OAuth) — default
conform integration ./my-plugin
conform integration ./my-plugin --auth oauth     # Explicit

# API key
export ANTHROPIC_API_KEY=sk-ant-...
conform integration ./my-plugin --auth api

# Or inline
ANTHROPIC_API_KEY=sk-ant-... conform integration ./my-plugin

Lint Rules

Validates across all plugin components:

• Manifest — required fields, schema compliance (plugin.json)
• Skills — folder structure, SKILL.md presence, frontmatter, description quality, naming (kebab-case), reserved words
• Commands — frontmatter or # Title heading, description field, file naming
• Agents — frontmatter fields (model, tools, description), kebab-case naming, model validation
• Hooks — nested format enforcement, valid event types, script existence, type field validation
• Schema — JSON Schema validation for plugin.json and marketplace.json
• Misplaced files — detects files/folders in wrong locations
• Cross-plugin — duplicate skill/command name detection across multiple plugins

Component Filters

Test specific components instead of everything:

conform structural ./my-plugin --skills            # Only validate skills
conform structural ./my-plugin --hooks --commands   # Hooks and commands only
conform structural ./my-plugin --agents             # Only validate agents
conform lint ./my-plugin --skills                   # Lint skills only

Available filters: --skills, --commands, --hooks, --agents

When no filter is specified, all components are validated.

Integration Options

conform integration ./my-plugin --auth api           # Use API key
conform integration ./my-plugin --auth oauth         # Use Claude CLI login
conform integration ./my-plugin --model sonnet       # Model selection (default: haiku)
conform integration ./my-plugin --max-turns 10       # Max conversation turns (default: 5)
conform integration ./my-plugin --timeout 120        # Per-test timeout in seconds (default: 60)
conform integration ./my-plugin --dry-run            # Preview test plan, no API calls
conform integration ./my-plugin --verbose            # Show full model responses
conform integration ./my-plugin --stop-on-fail       # Bail on first failure

Response Log Mode

Response-log mode runs user-defined prompts against your plugin via claude -p and captures the full responses. Use it to verify that your skills and agents produce the expected output when triggered by real prompts.

Skills and Agents Only

Response-log works with skills and agents — components that are triggered by natural language prompt matching. Commands (slash commands like /deploy) are user-initiated actions and can't be triggered through claude -p the same way.

For command testing, use integration mode instead. Integration tests use structured prompt templates that ask the model to self-report whether a command was loaded into its context (e.g. [COMMAND_LOADED:deploy]), without actually executing the command. This verifies that claude -p --plugin-dir correctly loads the command definitions.

Creating test-prompts.json

Place a test-prompts.json file in the root of your plugin directory. The file is a JSON array of prompt objects:

[
  {
    "id": "skill-data-analysis-basic",
    "component": "skill",
    "name": "data-analysis",
    "prompt": "I need to analyze data from a CSV file and create a chart showing the trends",
    "expect": {
      "responseContains": ["data", "trend"],
      "responseNotContains": ["error"]
    }
  },
  {
    "id": "agent-reviewer-trigger",
    "component": "agent",
    "name": "code-reviewer",
    "prompt": "Review the changes in this PR for security issues",
    "expect": {
      "responseContains": ["security"]
    }
  }
]

Prompt Fields

| Field | Required | Description | |-------|----------|-------------| | id | yes | Unique identifier for the test case | | component | yes | Component type: skill or agent | | name | yes | Name of the component being tested | | prompt | yes | Natural language prompt sent to claude -p — should match the component's trigger description | | expect | no | Optional expectations to validate against the response |

Expectations

The expect object supports these checks:

| Field | Type | Description | |-------|------|-------------| | responseContains | string[] | Response must contain all of these strings (case-insensitive) | | responseNotContains | string[] | Response must not contain any of these strings (case-insensitive) |

When no expect is defined, conform checks that a non-empty response was returned.

Verdicts

Each prompt gets a verdict:

| Verdict | Meaning | |---------|---------| | pass | All expectations met (or response received with no expectations) | | fail | One or more expectations failed | | warn | Component not loaded, or empty response with no expectations | | error | Process error (timeout, auth failure, CLI not found) | | skip | Dry-run mode |

Running

conform response-log ./my-plugin                     # Auto-discover test-prompts.json from plugin
conform response-log ./my-plugin --prompts t.json    # Use an external prompts file
conform response-log ./plugins/                      # Run across all plugins with test-prompts.json
conform response-log ./my-plugin --dry-run           # Preview plan without API calls
conform response-log ./my-plugin --verbose           # Show full model responses
conform response-log ./my-plugin --model sonnet      # Use a specific model
conform response-log ./my-plugin --timeout 120       # Per-prompt timeout (default: 60s)
conform response-log ./my-plugin --stop-on-fail      # Stop on first failure

Results are written to a timestamped JSON file in the report directory (./reports/ by default).

Report Options

Generate reports after test runs:

conform ./my-plugin --report                         # HTML report (default)
conform ./my-plugin --report --report-type md        # Markdown report
conform ./my-plugin --report --report-type json      # JSON report
conform ./my-plugin --report --report-dir ./out      # Custom output directory (default: ./reports)
conform ./my-plugin --report --report-open           # Open HTML report in browser

CI / JSON Output

conform lint ./my-plugin --json                      # JSON lint output
conform structural ./my-plugin --ci                  # Non-interactive output (no TUI)
conform ./my-plugin --ci --stop-on-fail              # CI mode, bail on first error
conform ./my-plugin --ci --json                      # JSON output for pipeline parsing

Configuration

Optional conform.yml in the plugin directory or working directory:

maxDescLength: 1024
rules:
  disable:
    - hooks/script-exists

Or pass config inline:

conform lint ./my-plugin --max-desc-length 2048      # Override max description length
conform lint ./my-plugin --disable hooks/script-exists,schema/marketplace
conform lint ./my-plugin --config ./custom-conform.yml

All Options Reference

| Flag | Description | Default | |------|-------------|---------| | --skills | Only validate skills | all components | | --commands | Only validate commands | all components | | --hooks | Only validate hooks | all components | | --agents | Only validate agents | all components | | --skip <list> | Comma-separated directory names to skip | — | | --auth <mode> | Auth mode: api or oauth | auto-detect | | --model <name> | Model for integration tests | haiku | | --max-turns <n> | Max conversation turns per test | 5 | | --timeout <secs> | Per-test timeout in seconds | 60 | | --verbose | Show full model responses | off | | --dry-run | Preview test plan without API calls | off | | --stop-on-fail | Stop after first failure | off | | --report | Generate a report file after tests | off | | --report-type <type> | Report format: html, md, json | html | | --report-dir <path> | Directory to write reports | ./reports | | --report-open | Open HTML report in browser | off | | --prompts <path> | Path to test-prompts.json | auto-discover | | --json | Output as JSON (lint and CI modes) | off | | --ci | Non-interactive output for CI pipelines | off | | --max-desc-length <n> | Max skill description length | 1024 | | --disable <rules> | Comma-separated rule IDs to disable | — | | --config <path> | Path to conform.yml config file | auto-discover | | --help, -h | Show help message | — | | --version, -v | Show version | — |

License

MIT