@desenlabs/conformance — DESEN Conformance Suite (Beta)

This package is the beta companion to SPEC.md 1.0.0-RC1. It contains fixtures that any conforming DESEN Runtime, DSL Producer, or Governance Core should pass. The suite is non-normative: failing a fixture does not in itself violate the spec, but fixtures in this directory are designed to catch concrete, implementer-blocking misreadings of the spec.

The final 1.0.0 release will ship a stable version of this suite with versioned fixtures, a signed manifest, and a reference test harness. This beta exists so that RC1 implementers can self-check without waiting.

Install

npm install --save-dev @desenlabs/conformance
# or
pnpm add -D @desenlabs/conformance

Directory layout

packages/conformance/
├── README.md                  ← this file
├── HARNESS.md                 ← runner internals + dispatcher contract
├── MANIFEST.yaml              ← machine-readable list of all fixtures
├── KNOWN_FAILURES.json        ← xfail registry (every entry must link a tracking issue)
├── run.mjs                    ← `desen-conformance` CLI + `runConformance({ bucket })` API
├── validation/                ← DSL schema conformance (§2)
│   ├── positive/              ← documents that MUST validate
│   └── negative/              ← documents that MUST NOT validate
├── runtime/                   ← fail-closed behavior (§3)
├── telemetry/                 ← envelopes and payloads (§4)
└── governance/                ← classifier, matrix, audit (§5)

Each fixture directory has its own short README that explains the bucket and lists what a correct implementation should do.

How to use this suite

You have three paths:

1. CLI. Once the package is installed, run npx desen-conformance (or desen-conformance --bucket=runtime for one bucket). The CLI exits non-zero on the first real divergence from the SPEC.
2. Programmatic API.

   import { runConformance } from "@desenlabs/conformance";
   const result = await runConformance({ bucket: "validation" });
   //  → { suite_id, suite_version, passed, failed, xfailed, unexpectedPassed, elapsed_ms }

   This is what alternate Runtime / Producer / Governance Core implementations should call from their own test runners.
3. Manual walk-through. Read each subdirectory's README, run its fixtures through your implementation, compare with the expected outcomes. This forces a slow read of each §.

Scope

The suite deliberately does not test:

• Specific visual rendering (CSS, layout engines). DESEN is modality-agnostic.
• Network or transport protocols between Runtime and data sources.
• The physical storage of audit logs or telemetry streams.

It does test:

• That every schema in §2 accepts positive fixtures and rejects negative fixtures with the correct failure mode.
• That a Runtime emits the right telemetry envelopes in the right sequence for known interactions.
• That the risk classifier (§5.2.1) returns exactly the expected R-Level for every diff pair.
• That the Authority × Risk matrix (§5.3) gates correctly.
• That audit chains verify, and that broken chains are detectable.
• That guardrails (§3.4) fail-closed with the correct error_class.

Reporting a fixture bug

If a fixture seems incorrect, open an issue against the spec repository with:

• The fixture path.
• The spec paragraph you believe it misreads.
• Your expected outcome and the suite's current expected outcome.

We would rather fix a fixture than accept a silent spec drift.

Stability

• Beta fixtures MAY change between RC1 and 1.0.0.
• At 1.0.0 the suite freezes. Post-1.0.0 changes follow the spec's SemVer rules (SPEC §7.2): additive in minor, breaking only in major.