Norma

The Spec Fidelity layer for AI-generated software.

Measure whether the code an AI agent built actually matches the spec it was given.

An auditor checks your code against the world's standards. Norma checks it against yours. It reads your spec, reads your implementation, and returns one number, the Spec Fidelity Score, plus a requirement by requirement breakdown and the exact location of every gap.

Other tools help you write specs and generate code. Norma tells you if the generated code kept its promises.

Quick start

npm install -g @norma-sh/norma

# bring your own key — any of these works
export ANTHROPIC_API_KEY=sk-ant-...   # default provider
# export OPENAI_API_KEY=sk-...        # then: norma check --provider openai
# export GEMINI_API_KEY=...           # then: norma check --provider gemini

norma check

No install needed? Run it once with npx @norma-sh/norma check. Works with Anthropic, OpenAI, Gemini, Groq, OpenRouter, Mistral, xAI, or a local Ollama — see docs/PROVIDERS.md.

Norma auto detects your spec (specs/, spec.md, AGENTS.md, CLAUDE.md, or PRD.md), reads your code, and prints a score:

  Spec Fidelity Score  78/100
  4 requirements  2 met  1 partial  0 missing  1 contradicted
  contradictions present, review before shipping

  drift:
  ⚠ R004 Rate limit is 100 requests per minute (src/limit.ts)
  ~ R002 Users can reset their password by email (src/users.ts)

That is the whole loop. No account, no setup, no other users required.

What the score means

Norma splits your spec into atomic requirements, judges each one against the code, and aggregates the verdicts:

| Verdict | Counts as | Meaning | | --- | --- | --- | | met | full | implemented and consistent with the spec | | partial | half | implemented incompletely or with caveats | | missing | zero | no implementation found | | contradicted | zero | implemented in a way that conflicts with the spec |

Requirements are weighted by priority (must, should, may), so dropping a must costs more than dropping a may. A contradiction always raises a flag, never just a quiet low score.

Usage

norma check [path]            # score a project (defaults to the current dir)
norma check --pr main...HEAD  # score only the files changed in a range
norma check --format json     # machine readable, for CI
norma check --threshold 85    # exit non-zero if the score is below 85
norma fix                     # explain each finding, recommend a fix, and write a repair prompt
norma fix --prompt            # output only the AI repair prompts, one per finding
norma fix --json              # the full remediation report as JSON
norma check --provider gemini # judge with another vendor (anthropic | openai | gemini | groq | openrouter | mistral | xai | ollama)
norma providers               # list providers, env vars, and which keys are set
norma init                    # write a norma.config.json
norma report                  # re-render the last run
norma login sk-norma-...      # store the dashboard key permanently (setx on Windows, ~/.norma elsewhere)
norma logout                  # remove the stored key
norma status                  # check whether the CLI is connected to the Norma dashboard

Every command also prints a one-line connection banner at startup — dashboard: connected (...) or dashboard: not connected (set NORMA_API_KEY ...) — so you always know whether report upload is active without running a separate command.

norma login stores the dashboard key durably — in the Windows user environment (via setx, no admin) or ~/.norma/credentials.json (mode 600) on macOS/Linux — and every later run picks it up automatically when NORMA_API_KEY is not set in the shell. An environment value always takes precedence, so CI configuration stays explicit.

norma fix reuses the same analysis as norma check: it scores the project, then for every requirement that is not met it generates a plain-language explanation, ordered fix steps, and a repair prompt you can paste straight into Claude Code, Cursor, Windsurf, GitHub Copilot, or Bolt. Remediation never changes the score.

Every run writes norma-report.json, the canonical artifact, conforming to schema/report.schema.json.

Configuration

norma.config.json at the project root:

{
  "spec": ["specs/**/*.md", "AGENTS.md"],
  "include": ["src/**"],
  "exclude": ["**/node_modules/**", "**/dist/**"],
  "threshold": 80,
  "model": "",
  "provider": "anthropic"
}

An empty model uses the provider's default. Full reference: docs/CLI.md; provider setup: docs/PROVIDERS.md.

Managed trial mode

By default the anthropic provider calls Anthropic directly with your own ANTHROPIC_API_KEY. A Trial team can instead run verifications key-free through Norma's bounded managed-inference proxy by setting:

export NORMA_MANAGED=1                 # route the anthropic provider through Norma
export NORMA_API_KEY=sk-norma-...      # your Norma API key (sent as a Bearer token)
export NORMA_API_BASE=https://app.norma.sh   # optional; defaults to https://app.norma.sh

In this mode the provider talks to ${NORMA_API_BASE}/api/managed/v1/messages instead of Anthropic; Norma uses its own server-side key and meters usage. The trial is capped (default 25 verifications). Once the cap is reached the proxy returns HTTP 402 and the CLI prints the limit message and stops — add your own provider key (ANTHROPIC_API_KEY, unset NORMA_MANAGED) to keep going. This is the one mode that routes through a Norma server; the default BYOK flow does not (see Privacy). The proxy is gated server-side and off until launch.

How it works

1. Extract. The spec is parsed into atomic, testable requirements with stable IDs.
2. Index. The codebase is indexed and, per requirement, the relevant files are retrieved.
3. Judge. Each requirement gets a verdict, from deterministic checks where possible and an LLM judge otherwise. The judge is model agnostic behind a provider interface, so Norma stays neutral.
4. Score. Verdicts are weighted and aggregated into the Spec Fidelity Score. The model and prompt version are stamped on every report so a score can be reproduced or contested.

Privacy

Norma runs locally. Code is read on your machine, and only the slices needed for judging are sent to your configured model provider. Bring your own key. Nothing is sent to Norma servers, because there are none. Files matching the secret patterns in the default exclude list (.env files, private keys, and similar) are never indexed or sent.

GitHub Action

Score every pull request and comment the result. Add this workflow to your repo:

name: Norma
on: pull_request
permissions:
  contents: read
  pull-requests: write
jobs:
  spec-fidelity:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - uses: norma-sh/norma@v1
        with:
          spec: "specs/**/*.md AGENTS.md"
          threshold: "80"
          pr-range: "origin/${{ github.base_ref }}...HEAD"
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}

The action posts a single sticky comment with the Spec Fidelity Score and the drift, updates it on each push, and fails the check when the score is below threshold. A full example is in examples/norma-pr.yml.

To judge with another vendor, set provider and the matching key input, e.g. provider: "openai" with openai-api-key: ${{ secrets.OPENAI_API_KEY }} (also available: gemini-api-key, groq-api-key, openrouter-api-key, mistral-api-key, xai-api-key).

MCP server (experimental)

Norma ships a local MCP server so an AI coding agent can verify its own output against the spec and pull a repair prompt without a human in the loop. It exposes two tools, norma_check and norma_fix, that map to the CLI commands of the same name. It is a thin adapter over the same library the CLI uses, so scoring, extraction, and the secret exclude list are identical, and your code never leaves the machine.

Build it, then register the norma-mcp bin with any MCP-capable client (Claude Code, Cursor, Windsurf). Until the package is published, point the client at the built file directly:

{
  "mcpServers": {
    "norma": {
      "command": "node",
      "args": ["/absolute/path/to/norma/cli/dist/mcp/server.js"],
      "env": { "ANTHROPIC_API_KEY": "sk-ant-..." }
    }
  }
}

Any provider key works here (OPENAI_API_KEY, GEMINI_API_KEY, ...) together with the provider tool argument.

Both tools take the same optional inputs as the CLI: path (defaults to the working directory), spec, pr, threshold, model, provider, and report. The server uses your own API key from the environment and only ever reads the filesystem; it does not modify files, it returns the fix for the agent to apply.

To also send each run's report to the Norma dashboard, create a key in the dashboard under API keys and add "NORMA_API_KEY": "sk-norma-..." to the env block — it is an upload-only key, separate from the model key. If you already ran norma login, the server falls back to the stored key and the env entry can be omitted. The full guide, including the key model, dashboard upload, and troubleshooting, is in docs/MCP.md.

This is experimental and is not published yet. Do not expose it more widely until judge accuracy has been checked by hand against real projects, since the server hands the judge to any agent that connects. See STATUS.md.

Status

This is the v1 alpha: the score engine, the CLI, and the GitHub Action. The Claude Code slash command builds on the same core and the same report schema.

Testing

npm test            # unit and integration, no API key needed

Judge accuracy is checked by hand against real projects — there is no automated accuracy harness. See EVALS.md for the testing strategy.

License

MIT