@xyo-network/xl1-statement-graph-sdk
v0.7.1
Published
Chain-facing SDK for the XYO Statement Graph — finalized-scan replay source, verified writer, and replay-contract checkpointing over structural XL1 viewer/datalake interfaces.
Keywords
Readme
@xyo-network/xl1-statement-graph-sdk
Chain-facing SDK for the XYO Statement Graph (see the repository WHITEPAPER). Everything chain-shaped is a STRUCTURAL interface (FinalizedBlockViewer, BodyStore, StatementBroadcaster) — a real XL1 gateway viewer and REST datalake runner satisfy them directly, and the testing package's fakes exercise every behavior deterministically.
Version note: the resume-integrity behavior described below (the floor bound, the checkpoint refusals and their details, the scan-side resume guards, the duplicate-transaction decode check, and the through RangeError) shipped in 0.6.0, published 2026-10-08. A consumer pinned to ~0.5.x (or a ^0.5 peer range) does not pick up 0.6.0 and keeps the 0.5.4 behavior until it moves to ~0.6.x. The strict gate profile described below (ReplayContract.gateProfile, its comparison on resume, and strict revision 1) shipped in 0.7.0, published 2026-10-09; a consumer pinned to ~0.6.x keeps the 0.6.0 behavior until it moves to ~0.7.x.
Bridge:
decodeHydratedBlock— XL1 hydrated blocks → committed-order abstract transactions plus non-transaction block carriers, with the D7-critical rule thattxIndexcounts the FULL committed block payload array before filtering. It returnsundefinedfor an unrecognizable or consensus-invalid block, including one that commits the same transaction twice.Scan (
scanFinalizedStatements): forward finalized walk with same-block elevated Claim hydration. Every transaction Bound Witness and every carrier consumed by the scan is canonical root-hash verified: required elevated Claims, configured auxiliary items, and co-committed objects named by verified Claims when strict body/schema equality is enabled. Unrelated non-transaction carriers do not affect Statement Graph replay.ClaimCarrierIntegrityErrorstops scanning on missing/mismatched Claim carriers, relevant non-Claim mismatches raiseBlockCarrierIntegrityError, and the datalake is never a Claim-event fallback. Verified object/auxiliary carriers take precedence over off-chain resolution. A height for which the gateway returns no block, a block for another height, or a block that fails decoding (shape, commitment, or duplicate-transaction checks) raisesBlockUnavailableErrorrather than being skipped; a gateway read that throws propagates its own error. Bulk range completeness, per-block fallback, head-regression checks against a resumed checkpoint'sthroughBlock,AbortSignal, strict object-body verification,indexedSchemaPolicy, and auxiliary candidates remain supported.Writer:
buildClaim({source, subject, hashes}), sharedprepareStatements,insertVerified(insert → readback hash check), andpublishStatements— Claims occupy only the elevated/on-chain partition and use XL1 root hashes; referenced object identity remains dataHash-based. Every named object must be co-committed or the whole Claim transaction is rejected. Objects whose client metadata makes root hash differ from dataHash are rejected because they cannot satisfy that commitment. The on-chain/off-chain transport partitions are disjoint and order-independent.Checkpoint: format version 2,
ReplayContract-scoped, with observable load causes (missing/corrupt/contract-mismatch). Stored and directly suppliedresumeFromvalues pass the same runtime parser before records or horizons are trusted: the scan throws oncorruptandcontract-mismatch, and rescans fromfloorBlockonmissing. AloadCheckpointcorruptdetail quoted below reaches a scan caller asresume checkpoint is corrupt: <detail> (Whitepaper §8.2). Legacy, foreign, and invalid checkpoints:- format 1 (written by releases before 0.5.0 other than 0.4.0/0.4.1), no marker, or any other marker below 2 or not a number loads as
missingand rescans from the floor; - 0.4.0/0.4.1 checkpoints are format 2 with the old
recordsshape, so they load ascorruptwithpre-fold-4 checkpoint (0.4.0/0.4.1 `records` shape); delete it to rescan from floor, and the scan throws. Delete them to rescan from the floor; - a format number above 2 was written by a newer SDK and loads as
corruptwithunsupported checkpoint format N; upgrade the SDK, so from 0.6.0 this SDK never silently rescans over it (0.5.4 and earlier still load it asmissingand rescan); - a fold-4 checkpoint that retains unresolved Claim references loads as
corrupt, and the scan requires a full rescan; - a checkpoint whose contract differs from the expected one in any field except
throughBlockiscontract-mismatch(contractMismatchnames the first field). From 0.7.0 that includes the gate profile, compared by value withgateProfileMismatchunder the legacy rule (see Strict gate profile): a profile-less checkpoint is accepted unless the expected profile setsrequireBodySchemaEquality. A caller that runsloadCheckpointitself must pass the effective contract, with the profile its scans run under: a profile-bearing checkpoint iscontract-mismatchagainst a contract without one; floorBlockis a hard lower bound on claim state. A frozen record whosefirstBlockprecedesfloorBlockloads ascorruptwithclaim record firstBlock F precedes floorBlock N; full rescan from floor required, and a watermark below the floor with the matchingwatermark block B precedes …detail.assembleCheckpointRecordsapplies the record check to every frozen half it assembles, inline or sharded, and prefixes a shard half's detail with its position (record shard I: claim record firstBlock …). That shard detail reaches only callers that resolve shards themselves: the scan refuses a sharded checkpoint first (see Resume). Delete the checkpoint and rescan from the floor.
The generic protocol still models unresolved event bodies for non-XL1/local backings, but finalized scans turn missing elevated carriers into fatal integrity errors.
- format 1 (written by releases before 0.5.0 other than 0.4.0/0.4.1), no marker, or any other marker below 2 or not a number loads as
Resume (
ScanOptions.resumeFrom): the scan walks[max(floorBlock, throughBlock + 1), head], so it never re-reads a walked block and never reads below the floor. A checkpoint sealed while the horizon was still below the floor (itsthroughBlockis belowfloorBlockand nothing was walked) resumes at the floor. Every resume check runs before any block is read:- the scan consumes only self-contained checkpoints. One that externalizes records to
recordShardsthrowsresume checkpoint externalizes its records to shards; assemble them before scanning, and inline records that fail assembly throwresume checkpoint records do not assemble: <detail>; - claim state (frozen records or
watermarkGroupEventKeys) without a foldwatermark, or awatermark.blockbeyondthroughBlock, is not a finalized-scan checkpoint and throws withfull rescan from floor required. An empty checkpoint without a watermark still resumes.loadCheckpointleaves these shapesvalidbecause other producers (sharded or local-log checkpoints) hold them legitimately; do not usecheckpointSeedas a fold seed for such a checkpoint; - a finalized head below
throughBlockthrowsFinalizedHeadRegressionError; - a caller
throughbelowthroughBlock, with the chain not regressed, is a request error: it throwsRangeError(through T is below the resume checkpoint's walked horizon W). Athroughabove the finalized head is still clamped to the head.
- the scan consumes only self-contained checkpoints. One that externalizes records to
Strict gate profile (
ReplayContract.gateProfile, 0.7.0; Whitepaper §8.2). The strict gates are dataset identity, declared in the contract as{rejectAmbiguousSchemaDeclarations, requireBodySchemaEquality, revision}and omitted for the default profile (both flags off), so default-profile contracts and checkpoints are byte-identical to 0.6.0.- Effective contract. The scan runs under
contract.gateProfile, or, when the contract has none, the profile the deprecatedScanOptions.strictdeclares atSTRICT_GATE_REVISION.ScanOptions.strictis still honored, so existing callers keep working, but new code declares the profile in the contract. A declared profile at another revision throwsreplay contract gateProfile revision N does not match this implementation (1); then, when both are present and declare different profiles, the scan throwsTypeError: ScanOptions.strict conflicts with contract.gateProfile; …, both before any read. The two are compared symmetrically by value, so an explicit all-off contract profile agrees only with all-off strict flags. The effective contract, withgateProfileonly when it is not the default, drives the resume check, extraction, strict body resolution, and the checkpoint the scan seals. - Resume. A checkpoint that records a profile resumes only under the same profile, by value and revision; under any other, the default included, the scan throws
resume checkpoint belongs to a different dataset: gateProfile … != …. Legacy rule: a profile-less checkpoint, written under the default profile or by 0.6.0 or earlier, is accepted unless the effective profile setsrequireBodySchemaEquality(gateProfile default (or written before gate profiles) != {…}; strict body-schema equality requires a full rescan from floor). The rule encodes revision 1, which leftrejectAmbiguousSchemaDeclarationsunchanged: under a profile at any other revision a profile-less checkpoint is refused (… strict revision N requires a full rescan from floor). A strictrejectAmbiguousSchemaDeclarationscheckpoint from 0.6.0 or earlier therefore resumes without a rescan, and the next checkpoint records the profile. SDK releases before 0.7.0 reject a profile-bearing contract and load a profile-bearing checkpoint ascorrupt, so a fleet that shares one checkpoint store upgrades together and cannot downgrade past a profile-bearing checkpoint. - Helpers.
canonicalGateProfile(profile)returns the canonical form (undefinedwhen both flags are off, at any revision).gateProfileFromStrict(strict)turnsStrictModeOptionsinto a canonical profile atSTRICT_GATE_REVISION; use it to declare the profile in the contract when migrating offScanOptions.strict, and in the contract you pass toloadCheckpoint(it returnsundefinedfor the default profile, whose key the contract omits).gateProfileMismatch(actual, expected)compares a checkpoint's profile (actual) with the expected one by value under the legacy rule, so it is asymmetric, and returns the mismatch detail orundefined;contractMismatchcalls it after every other field. - Strict revision 1. Under
requireBodySchemaEqualityan object body must be a plain record whoseschemais a string equal to the declaration (object-schema-mismatchotherwise; anything but a plain record, or no verified body, isobject-body-unverifiable), and a pure object whose hash isH({schema: declared})self-verifies with no stored body (withPureObjectBodies, from the protocol package). The scan applies it per transaction before extraction; auxiliary candidates keep the bodies actually read. A dataset folded underrequireBodySchemaEqualitybefore 0.7.0 needs a full rescan from its floor, which the legacy rule enforces. - Availability. Under
requireBodySchemaEquality, an object with no same-block carrier whose off-chain body is missing or fails its hash check is dropped for that entry (object-body-unverifiable; a failed hash also appears incorrupt), and the checkpoint still advances past the block. The exception is a correctly declared pure object, which self-verifies: it folds whether its readback is missing or corrupt (a corrupt readback is still reported incorrupt). Resume never revisits it, so the drop is permanent for that checkpoint lineage; only a rescan from the floor evaluates it again. This is decided behavior: object availability under strict mode is settled once per lineage (Whitepaper §8.4). Declaring the profile does not declare the object corpus, and strict datasets can see a different negation set from default ones (Whitepaper §14.1).
- Effective contract. The scan runs under
Account-mode transaction building (buildTransaction + addTransactionToChain) stays at the consumer boundary for now — publishStatements targets the universal addPayloadsToChain path both browser and Node gateways expose.
License
LGPL-3.0-only © XY Labs / XYO Foundation
