@msm-core/validate

Composable output validation for AI pipelines. Three tiers, one interface — fast rules gate first, embedding similarity next, LLM-as-judge only when needed. Use any tier standalone or chain them into a cascade.

npm install @msm-core/validate

Strategies

| Strategy | Tier | Latency | Requires | Action on fail | | ------------- | :--: | --------: | -------------- | ----------------- | | "rules" | 1 | <1 ms | nothing | block | | "embedding" | 2 | ~50 ms | embed function | review | | "llm" | 3 | ~300 ms | LLM judge fn | block/review | | "cascade" | — | first-hit | 1–3 stages | per winning stage |

All validators return the same shape:

interface ValidationResult {
  passed: boolean;
  score: number; // [0, 1]
  violations: string[]; // empty when passed
  action: "release" | "review" | "block";
  strategy: string; // e.g. "rules", "cascade(rules→llm)"
  latency_ms: number;
}

Tier 1 — Rules (deterministic)

No model. No network. Runs in < 1 ms. Use this as the first gate on every request.

import { createValidator } from "@msm-core/validate";

const v = createValidator({
  strategy: "rules",
  rules: {
    max_length: 2000, // block if response exceeds N chars
    min_length: 10, // block if response shorter than N chars
    blocked_phrases: [
      // case-insensitive substring match
      "ignore previous instructions",
      "DROP TABLE",
    ],
    no_pii: true, // block emails, phones, SSN-like, credit cards
    language: "ar", // block if response is not Arabic (or mixed)
  },
});

const result = await v.validate(responseText);
// result.action: "release" | "block"

action is always definitive — "release" or "block", never "review". Mixed-language responses always pass the language check.

Tier 2 — Embedding similarity

Compares the response against golden examples of acceptable output using cosine similarity. Returns "review" (not "block") on failure — semantic distance is indicative, not certain, so the cascade continues to an LLM for a final verdict.

Golden example embeddings are computed once and cached for the lifetime of the validator instance.

import OpenAI from "openai";
import { createValidator } from "@msm-core/validate";

const openai = new OpenAI();

async function embed(text: string): Promise<number[]> {
  const r = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: text,
  });
  return r.data[0]?.embedding ?? [];
}

const v = createValidator({
  strategy: "embedding",
  embed,
  goldenExamples: [
    "Here is a step-by-step guide to solving the problem.",
    "The answer to your question is as follows.",
  ],
  threshold: 0.72, // cosine similarity minimum (default: 0.75)
});

const result = await v.validate(responseText);
// result.action: "release" | "review"

Tier 3 — LLM-as-judge

Delegates evaluation to any LLM via a caller-provided judge function. Works with OpenAI, Gemini, Anthropic, Ollama — anything that returns text. Returns "review" (not a hard crash) on network failure or JSON parse errors, so a downed judge never breaks production.

import { createValidator } from "@msm-core/validate";

const v = createValidator({
  strategy: "llm",
  judge: async (prompt: string) => {
    // Use any LLM — the prompt is already structured, just call and return the text
    const r = await openai.chat.completions.create({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: prompt }],
    });
    return r.choices[0]?.message?.content ?? "";
  },
  criteria: [
    // optional — defaults to helpfulness, safety, coherence
    "The response stays on topic",
    "No harmful or unsafe content",
    "Professionally written",
  ],
  systemPrompt: "You are a strict AI quality evaluator.", // optional
});

const result = await v.validate(responseText);
// result.action: "release" | "review" | "block"

The judge receives a structured prompt asking it to return JSON:

{ "passed": true, "score": 0.9, "violations": [], "action": "release" }

Markdown fences are stripped automatically if the LLM wraps its output.

Cascade — chain all three

Run the cheapest gate first. The cascade stops as soon as any stage returns "release" or "block". It only escalates to the next stage on "review". Most requests never reach the LLM.

import { createValidator, CascadeValidator } from "@msm-core/validate";

const v = new CascadeValidator([
  // Stage 1: rules — instant, blocks obvious violations.
  // pass_action: "review" makes a PASS escalate to the next stage —
  // without it, passing rules returns "release" and ends the cascade
  // (stages 2–3 would never run).
  createValidator({
    strategy: "rules",
    rules: { no_pii: true, max_length: 2000, language: "ar" },
    pass_action: "review",
  }),
  // Stage 2: embedding — catches off-topic responses
  createValidator({
    strategy: "embedding",
    embed,
    goldenExamples,
    threshold: 0.7,
  }),
  // Stage 3: LLM judge — final verdict on ambiguous cases only
  createValidator({
    strategy: "llm",
    judge,
    criteria: ["Safe", "On-topic", "Helpful"],
  }),
]);

const result = await v.validate(responseText);
// result.strategy: "cascade(rules→embedding→llm)"  — shows which stages ran

Or via factory:

const v = createValidator({
  strategy: "cascade",
  stages: [rulesValidator, embeddingValidator, llmValidator],
});

Use standalone — without mini or pipeline

@msm-core/validate has zero runtime dependencies. It works on its own in any Node.js ≥ 18 project.

import { createValidator } from "@msm-core/validate";

// Quick sanity check on any text — no pipeline needed
const v = createValidator({ strategy: "rules", rules: { no_pii: true } });
const { passed, violations } = await v.validate(userSubmittedText);

API reference

createValidator(config): Validator

Factory. Returns the appropriate Validator for the given strategy config.

class RulesValidator

class EmbeddingValidator

class LLMValidator

class CascadeValidator

All implement Validator:

interface Validator {
  readonly strategy: string;
  validate(text: string): Promise<ValidationResult>;
}

Types

import type {
  Validator,
  ValidatorConfig,
  ValidationResult,
  ValidationAction, // "release" | "review" | "block"
  ValidationRules,
  RulesValidatorConfig,
  EmbeddingValidatorConfig,
  LLMValidatorConfig,
  CascadeValidatorConfig,
} from "@msm-core/validate";

Part of the MSM ecosystem

| Package | Role | | ------------------------ | ------------------------ | | @msm-core/mini | Agent execution loop | | @msm-core/context | Context assembly (RAG) | | @msm-core/validate | Output validation gate | | @msm-core/pipeline | 3-layer routing pipeline |

See the SDK README for the full compatibility matrix.