@promptsnap/diff

Layered diff engine for LLM outputs. Used internally by the PromptSnap CLI.

Layers (in order)

| Layer | Input | Verdict | |---|---|---| | Exact | Redacted text hash | equivalent (fast path) | | Canonical | Structured parse of JSON/markdown/tool-calls | changed / continues | | Embedding | Cosine similarity via Embedder | equivalent above threshold | | Judge | LLM judge call (Anthropic / OpenAI) | equivalent / changed / needs_review | | Abstain | Fallback when no judge configured | needs_review |

Usage

import { runDiffPipeline, AnthropicJudge, OpenAIEmbedder, JudgeCache } from '@promptsnap/diff';

const outcomes = await runDiffPipeline(inputs, {
  embedder: new OpenAIEmbedder('text-embedding-3-small'),
  embeddingThreshold: 0.97,
  judge: new AnthropicJudge('claude-3-5-haiku-20241022'),
  cache: new JudgeCache('./node_modules/.cache/promptsnap'),
  maxJudgeCalls: 50,
  concurrency: 4,
});

Calibration eval

Measures precision (≥ 0.92 on equivalent) and recall (≥ 0.85 on changed) against a 200-pair labeled set:

pnpm --filter @promptsnap/diff eval

Documentation

→ docs.promptsnap.live/reference/diff-engine