@zioladev/interop-conformance

Qualify WebMCP interoperability against frozen, versioned profiles.

@zioladev/interop-conformance takes already-recorded interoperability evidence — Phase III's provider-conformance-report/2 — and evaluates it against a frozen, versioned profile, producing a three-state result: qualified, not_qualified, or insufficient_evidence, bound to the exact evidence by content hash.

It judges; it does not execute. It runs no model, tool, or browser, and it never rewrites a lower layer's attribution — it reads the closed report artifact and applies frozen criteria. There is no universal "WebMCP compliant" verdict: qualification is always relative to a named profile.

Quickstart

import {
  qualify,
  TRAJECTORY_MULTI_PROVIDER_NATIVE_V1,
  renderQualificationResult,
} from '@zioladev/interop-conformance';

// `bundle` is an array of provider-conformance-report/2 objects (Phase III output, as parsed JSON).
const result = qualify(TRAJECTORY_MULTI_PROVIDER_NATIVE_V1, bundle, { qualifiedAt: new Date().toISOString() });

result.status;              // 'qualified' | 'not_qualified' | 'insufficient_evidence'
result.missing;             // required lanes / requirements that could not be evaluated
result.failures;            // requirements that definitively FAILED
result.requirements;        // every requirement's per-lane verdict: 'held' | 'failed' | 'unevaluable'
result.evidence;            // manifest: profile version + each report's version/generator/lane/hash

console.log(renderQualificationResult(result));

Profile: trajectory/multi-provider-native@1
Evidence lane(s): chrome-webmcp (Chromium/152)

  minProviders @ native-browser-lane            PASS  held for all trajectories
  terminalAttained @ native-browser-lane        PASS  held for all trajectories
  trajectoryConformance @ native-browser-lane   PASS  held for all trajectories
  providerGrade @ native-browser-lane           PASS  held for all trajectories
  carriedProvenance @ native-browser-lane       PASS  held for all trajectories
  noProviderOwnedFailure @ native-browser-lane  PASS  held for all trajectories

Status: QUALIFIED
Evidence:
  - @zioladev/[email protected] · provider-conformance-report/2 · order-then-drift/native-town · sha256:d086ce29bf7b…

The three states — and why insufficient_evidence is first-class

Missing evidence is never treated as pass or fail. The same native report qualifies the native profile but returns insufficient_evidence against the reference profile — because the reference lane simply isn't in the bundle:

qualify(TRAJECTORY_MULTI_PROVIDER_NATIVE_V1,    [nativeReport]).status;    // 'qualified'
qualify(TRAJECTORY_MULTI_PROVIDER_REFERENCE_V1, [nativeReport]).status;    // 'insufficient_evidence'  (missing: reference-lane)

That single flip — same evidence, different frozen profile, different result — is the whole point: profile-relative qualification, with no new execution.

Every requirement carries its own verdict — held, failed, or unevaluable. A requirement that cannot be computed from the evidence (e.g. carried-state provenance when the report records no carried values and no publish declarations) is unevaluable → insufficient_evidence. It is never silently mapped to false. A definitive failure (failed) takes precedence → not_qualified.

Profiles are frozen data

A profile is a closed vocabulary of required lanes + requirements — serializable, comparable, Object.freezed before any evaluation. Two ship in 4A, differing only by required lane:

| Profile | Required lane(s) | Requirements | |---|---|---| | trajectory/multi-provider-reference@1 | reference-runtime/* | ≥2 providers · terminal attained · conformance PASS · provider grade PASS · carried-provenance valid-if-present · no provider-owned failure | | trajectory/multi-provider-native@1 | chrome-webmcp + native document.modelContext | (identical) | | trajectory/multi-provider-dual-lane@1 | both reference and native | (identical, per lane) |

Deferred profiles (absence is intentional, not an omission)

Some profiles are deliberately not shipped because provider-conformance-report/2 does not yet carry the evidence they'd need. They are listed here so their absence is understood as honesty, not a gap — each will arrive when the named additive provenance field is added upstream (never inferred through a side channel):

• consumer/cross-model profiles — need the model id per report. /2 records it in the observation, not the assembled report; a matrix must not grow Claude/GPT/Gemini columns it cannot actually attribute. Deferred pending an additive modelId report field.
• trajectory/carried-state (required) and same-trajectory portability profiles — need the spec's publishes / requiredInputs declarations to tell "carried nothing by design" from "carry failed." Until then an empty carried[] under a required rule is honestly unevaluable, not a failure.

Composing evidence across reports

A profile can require several lanes; qualify() composes them from multiple reports into one result — without ever merging partial truths. Each required lane's entire requirement set is satisfied by a single report (or one group of same-target repeated runs); evidence is never stitched across reports to manufacture a claim no single report supports.

qualify(TRAJECTORY_MULTI_PROVIDER_DUAL_LANE_V1, [referenceReport, nativeReport]).status;  // 'qualified'

• Requirement→evidence mapping — each RequirementResult.evidence is the reportHash that produced its verdict, so you can see exactly which report satisfied which requirement.
• No cherry-picking — two same-target runs where one failed ⇒ not_qualified (a passing sibling can't mask a failure).
• Compatibility gate — a profile can demand the same trajectory / providers across lanes (crossLaneCompatibility); incompatible evidence ⇒ insufficient_evidence, never a fused claim.
• Bundle manifest — per-lane evidence refs plus an order-independent bundleHash.

The qualification matrix

buildQualificationMatrix(profile, bundle) structures a result into a requirement × required-lane grid — filling only the cells the profile requires, with each cell's evidence lineage — and derives the final status from profile logic (never a vote over cells).

import { buildQualificationMatrix, renderQualificationMatrix, TRAJECTORY_MULTI_PROVIDER_DUAL_LANE_V1 } from '@zioladev/interop-conformance';
console.log(renderQualificationMatrix(buildQualificationMatrix(TRAJECTORY_MULTI_PROVIDER_DUAL_LANE_V1, [referenceReport, nativeReport])));

requirement             reference-lane       native-browser-lane
minProviders            PASS                 PASS
terminalAttained        PASS                 PASS
trajectoryConformance   PASS                 PASS
providerGrade           PASS                 PASS
carriedProvenance       PASS                 PASS
noProviderOwnedFailure  PASS                 PASS

Evidence lineage:
  reference-lane       ⇐ order-then-pastry/3c-live · sha256:70272d51e3d3…
  native-browser-lane  ⇐ order-then-drift/native-town · sha256:d086ce29bf7b…
  bundle: sha256:3a13b5a882b3…

Status: QUALIFIED   (profile logic — a cell is evidence, not a vote)

A cell is evidence, not a vote. A failing required lane is not_qualified even when a sibling lane is fully green — green cells never outvote a disqualifying required cell.

The laws

• D1 — Profile-relative, never universal. No generic "WebMCP compliant" result.
• D2 — Missing evidence is not failure. It is insufficient_evidence; an unevaluable requirement is never false.
• D3 — Never rewrites lower-layer attribution. A model-owned failure stays model-owned; a profile may be not_qualified without relabeling it a provider failure.
• D4 — Evaluates evidence; does not execute. No model/tool/browser is run.
• D5 — Criteria are frozen before evaluation. Profiles are frozen data; requirements can't move after seeing a run.
• D6 — Qualification is bound to its evidence. The result carries an evidence manifest hashing each report; a new provider/browser/model/profile version requires re-qualification.
• D8 — Aggregation never merges partial truths. A lane is satisfied by one report (or one group of same-target runs), never by stitching requirements across reports; cross-lane composition is gated by compatibility.
• D9 — A matrix cell is evidence, not a vote. The final status follows profile logic; a disqualifying required cell disqualifies, and green cells never outvote it.

Evidence binding

A result means "qualified under profile X, against this exact evidence, by generator Z" — the EvidenceManifest records each report's reportVersion, generator + version, lane, provider ids, generatedAt, and a SHA-256 of its canonical content. Not a floating badge.

Package family

@zioladev/provider-tools        declare
@zioladev/provider-conformance  measure
@zioladev/interop-runtime       execute
@zioladev/interop-conformance   qualify   ← this package

Clean-room: imports nothing from any other package (a test asserts it) — it validates the report contract, it does not depend on the engine that produced it.

Scripts

npm run typecheck   # tsc --noEmit
npm test            # node --experimental-strip-types --test  (Node >= 22.6)
npm run build       # emit dist/ (ESM + .d.ts)

License

Apache-2.0. See NOTICE.