@unrulysystems/native-motion-conformance

The cross-engine conformance harness (SPEC-CONFORMANCE, private). One engine-agnostic scenario format, authored once, executed against two engines — the native core and the pinned web oracle ([email protected], see SEAMS.md) — then differentially compared within ratified tolerance bands. Its job is to make "the loop can never widen a band or silently skip a check" mechanical.

The accounting model (the load-bearing idea)

Every scenario run returns one of three verdicts — pass / fail / unassertable — and the RUNNER, never the scenario author, decides. aggregateSuite (src/suite.ts) folds all per-(scenario, engine) results fail-closed:

• any fail → suite fails; an empty result set → suite fails;
• unassertable is a ROUTED obligation, not a skip. Cross-engine-parity rows discharge in one of two ways (SPEC-CONFORMANCE §3, ratified Option A — R7 M5):
  • Scalar-comparable → owned in-suite. When the comparison reduces to a scalar, a real second engine is injected and its SAME-ID pass is ADDED via the parity seam (src/parity.ts), never edited in, so the row flips unassertable → owned without touching any scenario. gesture.cross-engine-parity works this way (native core vs an injected motion/react web engine).
  • External-evidence-only → stays unassertable, discharged elsewhere. When parity needs a real DOM/layout engine, real host components, or composite aesthetic judgment, no in-process engine can assert it, so the deterministic suite keeps the row unassertable and it is asserted at a NAMED external altitude — declared either as a frozen counterpart record (LAYOUT_PARITY_COUNTERPART / LAYOUT_IDENTITY_PARITY_COUNTERPART → the Playwright Chromium gate) or, for CMP-R7-cross-engine-parity, an owner-naming reason pointing at its executed legs (variants.e2e.ts + PROOF.md §R7), or for CMP-R8-cross-engine-parity, its executed Chromium web leg (keyframes.e2e.ts, 20/20) and executed M5 PROOF.md §R8 device keyframes card (Android real-touch verdicts + measured arcs; iOS AUTO-RUN 13/13 + measured peaks), or for CMP-R9-cross-engine-parity, its executed Chromium legs (examples.smoke.e2e.ts + text-layout.e2e.ts + image-gallery.e2e.ts, 22/22) and executed M5 PROOF.md §R9 device element cards (Android gallery 17/17 including stable image-gallery post-fix; iOS AUTO-RUN COMPLETE 17/17 with wallpaper loaded). appstore-choreography.cross-engine-parity is likewise external (SPRING goldens + Chromium). These never flip to an in-suite pass; routed > 0 → FAIL still binds.

Surface map

| Area | Role | Key files | | --------------- | ----------------------------------------------------------------------------------- | ------------------------------- | | scenario format | Pure-data timelines + invariants + declared capabilities | scenario.ts, scenarios/*.ts | | adapters | Per-engine determinism-fixed wrappers (native core; motion-dom under manual timing) | adapter.ts, adapters/ | | runner | Drives one scenario on one engine → three-valued verdict | runner.ts | | comparator | Pure cross-engine differential | comparator.ts | | suite | Fail-closed aggregation | suite.ts | | config | Ratified tolerance bands — never loop-tunable | config.ts | | authoring | Registry-backed capability gate at author time | authoring.ts | | parity | The additive cross-engine ownership seam | parity.ts |

Reader note: runSpringScenario/runValueTypeScenario mark per-engine readings pass: true by construction — for those kinds the real verdict is the cross-engine DIFFERENTIAL, not the single-engine run (runner.ts explains this inline).

Verification

113 tests / 17 files: 10 cross-engine conformance suites (L2's layout-identity and L4's appstore-choreography are the ninth and tenth), 5 harness unit suites (including the fail-closed accounting proofs), 1 wiring smoke, and motion-seams.canary.test.ts — canaries that pin the exact motion internals the harness drives, so a motion upgrade fails loudly BEFORE it can skew a verdict. scripts/gen-*-goldens.mjs (not CI gates) regenerate core's golden files from the pinned oracle on an intentional bump.

cd packages/conformance && bunx vitest run