@scientific-method/standard-checker

Checks a game restoration's spec/, parity/ and deviations/ against version 1 of the dinorefurb documentation standard, and writes what the standard says is generated: the four indexes in spec/index/, the totals in PARITY.md, and on request a validation run file in validation/. It has no dependencies and needs Node 22 or later.

pnpm add -D @scientific-method/standard-checker
pnpm exec standard-checker            # check, then rewrite stale indexes and PARITY.md
pnpm exec standard-checker --check    # check, and fail on a stale index or PARITY.md

It exits with 0 when the repository passes, 1 when it reports problems (one line per problem, starting with the file's path), and 2 when the options are invalid or --record-validation cannot write the record. --record-validation cannot be combined with --check, nor --require-ksc with --no-ksy, nor --message with either --check or --record-validation.

A pass ends with spec check passed: and the counts of entries, parity rows and deviations. When a step of the check did not run, it ends with spec check passed with skipped steps:, the counts, and Skipped: followed by each step and why, such as Skipped: Kaitai compilation of 2 definitions (--no-ksy). A failing run lists the skipped steps after its problems. A call or emit whose arguments cannot be counted, because the Parameters section it is counted against is missing or is not None. or a list of parameters, or because its argument list is never closed, is a skipped step as well. For a section in another form, the skipped step names the first thing that stops the count, such as item 2 names more than one parameter or it holds text and no list.

Without --base, the checker compares the spec with where HEAD forked from origin/$GITHUB_BASE_REF, or from origin/main when that variable is unset. Outside a git repository, in a shallow clone, or when that branch was never fetched, the fork point does not resolve and the comparison does not run. The result line then names it, such as Skipped: comparison with the base branch (HEAD has no merge-base with origin/main, fetch it with enough history or pass --base). Without git on PATH the skipped step names the missing git instead. With --require-base, either case is a problem and the run fails.

The same comparison finds an ID taken twice before the two branches meet. A spec ID or deviation whose file this change adds since the fork point, committed or not, fails when the tip of the base branch (--base, or origin/$GITHUB_BASE_REF or origin/main) holds a file under that ID that matches neither the working tree, the index, nor the file at any commit of this branch since the fork point: another change took the ID first, and IDENTIFIERS-6 has the branch merged second renumber its entry before it is merged, as in spec/findings/FND-SCORE-012.md: FND-SCORE-012 also exists at origin/main with content this branch never held; renumber this one before it is merged [IDENTIFIERS-6]. A tip that holds the branch's own entry, merged or cherry-picked there in an earlier version, passes, and so does one the base branch edited after taking it. Run the check before committing an entry, for example from a pre-commit hook, so the renumbering happens before commit messages and other files cite the ID. Pass the branch a change merges into as --base when it is not the default, such as a long-lived branch that several working branches merge into. Only the tip of that one branch is read: an ID taken on another branch, or in a draft no one has committed, is not seen until it reaches it.

A problem that breaks a numbered rule of the standard ends with the rule's label in brackets, such as [STATUS-14]. The standard opens that rule with the heading ###### STATUS-14, anchored at #status-14 on the site and in the copies restorations vendor, so the rule can be read on its own. Rules are numbered in Identifiers, Status and Entry types up to the end of Builds so far, and problems under other sections carry no label yet.

A build that keeps a listing record, spec/builds/<ID>.listing.yaml named by the entry's listing field, has it checked against its manifest and its list of other files (ENTRY-TYPES-18): every file the record lists is in the manifest with the manifest's size, in the list of other files, or under a directory exclusion; every manifest path and every listed other file is in the record; and every link or stopped path is in the list of other files by its own path. A disc read from an image names that image in source by a path the manifest or the list of other files gives (ENTRY-TYPES-16). Each problem names the path. Agreement shows only that the three name the same paths. It does not show that the game uses a file or that the listing missed nothing. Where the list of other files is prose in the Other files section, or <ID>.other-files.yaml could not be read or does not exist, the comparison with it is a skipped step on the result line that gives the reason, and only a readable <ID>.other-files.yaml is compared.

Each pattern in a format entry's files matches a file of the manifest of each build it lists. A superseded format entry keeps the files it had when it was replaced (IDENTIFIERS-7), so its pattern also passes when it matches a path that <ID>.other-files.yaml gives by its own path. A directory exclusion, a list written as prose and a list that could not be read or does not exist do not let it pass, and the problem names which of them stopped it.

Options

| Option | Meaning | Default | |---|---|---| | --root <dir> | The repository to check. | the current directory | | --check | Fail when an index or PARITY.md is stale, instead of rewriting it. | rewrite | | --scheduled-generation | For a restoration that updates the indexes and PARITY.md on its main branch only, such as from a scheduled job. Neither write nor compare them with the spec, and fail when the change since the base (where HEAD forked from --base, or the fork point) edits, adds or removes one, untracked files git does not ignore included. A file that matches its copy at the base branch's tip (--base, or origin/$GITHUB_BASE_REF or origin/main) passes, as does a change that only regenerates them and touches nothing else. The result line names the comparison with the spec as skipped. Without a base the change is not compared either, and the base comparison is named as skipped or, with --require-base, fails. | write or compare | | --base <ref> | The branch this one merges into. Fail when a spec ID, area or deviation that exists where HEAD forked from <ref> is gone, when a superseded format entry has no layout table although it had one there, or when an ID this change adds exists at <ref> with content this branch never held (see below). | origin/$GITHUB_BASE_REF or origin/main; when HEAD's fork point with it does not resolve, the comparison is named as skipped | | --squashed <list> | Comma-separated items OLD=NEW, or OLD=NEW+NEW for an entry replaced by several: superseded entries this change deleted, squashed into the replacements that keep the final content. Without it, a deleted ID fails IDENTIFIERS-6. A listed deletion passes when the entry's superseded_by at the base names exactly the listed replacements and each replacement exists now and is not superseded, or is listed too, so A=B,B=C squashes a chain. An OLD that still exists fails, so keeping the option after the squash reaches the main branch stops a squashed ID from being used again. One the base does not have is named as a skipped step. A squashed ID still cited in the spec, the glossary, the --code and --references directories, parity/ or deviations/ fails, a build or source alias included, and the message names the replacements to cite. The indexes and PARITY.md are written by the check, not searched for citations, so with --scheduled-generation they may name a squashed ID until the main branch regenerates them. Keep the list where the reference check does not read it, such as a workflow file or a text file a wrapper script reads: a .js, .mjs, .ts, .cs or .ps1 file under a checked directory that spells the list out cites every squashed ID in it and fails. | none | | --require-base | Fail when no --base is given and the fork point does not resolve, instead of passing with the comparison skipped. | pass with the comparison skipped | | --no-ksy | Skip compiling the Kaitai definitions in spec/formats/. The result line names the skipped compilation. | compile | | --require-ksc | Fail when spec/formats/ holds Kaitai definitions and no compiler is found, instead of passing with the compilation skipped. | pass with the compilation skipped | | --glossary <path> | Also accept the terms of a draft glossary file, or of a directory of them. | none | | --code <dirs> | Comma-separated directories whose files may cite spec and deviation IDs and hold PLACEHOLDER comments. | src,tests,tools | | --references <dirs> | Comma-separated directories whose files may cite IDs but whose PLACEHOLDER comments do not count against parity. | none | | --images <ranges> | Comma-separated half-open address ranges of the original's flat 32-bit images, such as 0x00400000..0x004C9000. A 0x value inside one that a code comment gives, or that the code uses under a comment, must be recorded in an entry the comment cites. | none, so only fn_ and g_ names are checked | | --max-range <bytes> | The largest address range an entry can record an address by. A larger one, such as a whole section, records only its two ends. | 0x10000 | | --data-dirs <dirs> | Comma-separated top-level directories of the original's data. A path into one must name, with its exact case, a file of some build's manifest or a path of some build's <ID>.other-files.yaml. A path under a directory exclusion counts only when the list gives it by its own path. A path that names nothing known is reported only when its last part holds a digit and no placeholder such as nn. | the top-level directories of the files the manifests list | | --rebuild <dirs> | Comma-separated directories that hold the rebuild. No Markdown file in spec/ may name a path in them, or a source file found in them by its file name. In a source entry in spec/sources/, a path counts only when it exists, in any case, so the entry can cite the external source's own paths. An empty value turns the check off. | src,tests | | --rebuild-source <dirs> | Comma-separated directories that hold the rebuild's source, which cites the rules, formats, screens and bugs it implements (see below). Only files that a --code directory also holds are read. An empty value turns the check off, and the address hint goes back to suggesting a finding. | src | | --message <file> | Check only the commit message in the file: every address it gives must be recorded in an entry it cites, as for a code comment. Everything from the scissors line of git commit --verbose on is left out; comment lines before it are checked, since git keeps them under git commit -m. Exits with 0 or 1, or 2 when the file cannot be read. For a commit-msg hook. | not checked | | --record-validation <builds> | Write validation/<date>-<commit>.md for the marked test files of validated parity rows, naming the comma-separated build IDs the run used and HEAD as the commit the run tested, and delete every other run file, except one that lists a test file git tracks but a sparse checkout does not hold. Run it only after every test in those files passed with none skipped, against HEAD as committed: it refuses, with exit code 2, when the working tree differs from HEAD in anything outside validation/, counting untracked files that git does not ignore. | not written | | --help | Print the options. | |

The work protocol's Implementation batches section has code in src/ cite the rules, formats, screens and bugs it implements and no findings or experiments. Each FND- or EXP- ID that a file under a --rebuild-source directory cites fails, once per file with the lines that cite it. The message names the candidates the protocol moves the citation to: the rules, formats, screens and bugs that are not superseded and list the finding or experiment in their evidence, or, for a superseded one, those its superseded_by leads to. For example: src/Game.cs: line 3 cites FND-SCORE-001, a finding, and code in the rebuild's source cites only the rules, formats, screens and bugs it implements; RULE-SCORE-001 lists it in its evidence: move the citation to it where the code implements it and the comment does not cite it yet, and delete it otherwise. Where there is no candidate, the message says so: the citation is deleted, and the code goes with its tests unless an entry or a deviation that is still live describes what it does. The check cannot tell which entries the code implements, so the choice among the candidates is the session's, and so is the row that each new citation sends back to Code partial. A superseded finding or experiment cited there is reported by this check alone, with its replacements, in place of the citation check's which is superseded; cite what replaced it. A finding or experiment that alone replaced a superseded rule, format, screen or bug is not reported: the protocol's research batch leaves the code written for that entry citing it, with the check passing, until an implementation session removes the code.

The address check's hint follows the same rule. For an address in a file of the rebuild's source that no entry the comment cites records, the message names the rules, formats, screens and bugs that record it or cite evidence that does (RULE-SCORE-001 records it or cites evidence that does: cite the one the code implements). Where none does, it says to leave the address out of the comment and to treat code that needs it as a gap in the spec, and suggests no finding. In other code directories, such as tests and tools, the hint still suggests the finding that records the address.

The KSC environment variable names the Kaitai Struct compiler. Without it, the checker looks for kaitai-struct-compiler or ksc on PATH. When it finds neither, it skips the compilation and names it in the result line, or with --require-ksc reports the missing compiler as a problem.

On Windows the search tries each name with the PATHEXT extensions, as cmd.exe does, so it finds the .bat launcher of the official release and skips the extensionless Unix script beside it. The checker then runs the launcher by its full path. A KSC that holds a bare name, with no directory, is looked up on PATH the same way.

On every platform, a launcher found on PATH whose --version fails is not used, and the checker prints a warning naming its path, with its output.

Coverage

standard-coverage measures how much of each analysed file of code the spec cites and has read completely, against the function inventories in coverage/, and how much of each file the inventory maps, as the work protocol's Measuring progress section describes. It keeps citation coverage, complete reading and mapping coverage apart, and gives no figure that adds files together.

pnpm exec standard-coverage                       # per-file figures and the uncited functions
pnpm exec standard-coverage --list                # also every function with the entries citing it
pnpm exec standard-coverage --json                # the same figures as JSON
pnpm exec standard-coverage --require-complete    # fail while an in-scope function is uncited

An inventory is coverage/<build>/<file>.tsv, with <file> the manifest's path and a CD: prefix written as a directory @CD. Its columns are start and size, then optionally name, out_of_scope and ranges. start is in the standard's notation for the file's format: an address, or for MZ overlay code an offset inside a row of the build's Code ranges. When the body is not one range from start, ranges lists its half-open ranges as start..end in the same notation, separated by spaces: 0x00401000..0x00401010 0x00401200..0x00401210. size is then their total, one of them holds start, and they do not overlap. In an NE file each range, and a body without ranges, ends in the segment it starts in.

Every file a manifest lists whose format, or for a packed file whose unpacked format, holds code and that has no inventory is printed as <build>: <file> has no inventory and is not measured, followed at the end by how many there are, and --json lists each under withoutInventory with build, file and format. No figure covers those files, so an empty unmeasured says only that every inventory present was read. Whether a file without an inventory is outside the work's scope is for the report that uses the figures to say: the manifest lists every file the game uses, including those of builds nobody has analysed. --require-complete measures the inventories that exist and does not fail on a file without one. Members of an archive are not listed, since the checker does not read a manifest's members yet.

An entry cites a function when one of its locations names the same build and file and its address, offset or half-open range overlaps the function's body: its ranges, or size bytes from start when the row gives none. Locations with kind: file-data, into the unpacked form of a packed file, or of superseded entries cite nothing, and neither does an address written in an entry's body. Real-mode segmented addresses (MZ, COM) are compared by the linear address they name, and each NE segment is a space of its own. A row without ranges is measured as if its body were contiguous. The shares count functions and bytes of the functions not out of scope.

Two rows may list the same bytes, as an analyzer that gives one chunk to several functions (a shared function tail) writes them. A location in those bytes cites every function that lists them, and the byte figures count each byte once: bytes and citedBytes are over the union of the bodies. A byte is cited when a cited function lists it, so citedBytes can equal bytes while a function whose whole body lies in a cited one's is still uncited; the function figures and --require-complete count that function. The bytes that two or more in-scope functions list are sharedBytes in --json, and the text line ends with ; <n> bytes listed by more than one in-scope function when there are any.

A cited function has been looked at, not read completely. A function counts as read completely when the locations of the static findings that entries list in complete_reading cover every byte of its body, together when several entries list them; a finding located only by the function's start covers one byte. Only the complete readings the standard accepts count (STATUS-4): static findings the entry lists in evidence and that list its first build, of an entry that is neither superseded nor disputed and whose procedure has no # may run: comment, and that are not superseded themselves. An entry's status counts for nothing here. The text prints complete reading: <n> of <m> functions, <b> of <B> bytes under each inventory, --json gives completeReading as functions and bytes, and --list marks each function read completely, with read in --json. The count is 0 when no complete reading covers a function. It is unavailable, <reason>, and completeReading is { "unavailable": "<reason>" } with no read in the list, when loading the spec had problems, since an entry that could not be read may hold a complete reading, and for a file where a location of a counted finding does not parse. ADR 0043 gives the reasons.

The .provenance.tsv and .regions.tsv files the work protocol puts beside an inventory are read when they exist; a missing one is no problem. The provenance file holds one name and value per line, separated by a tab, each name once, and must give xxh3. An inventory whose provenance gives another xxh3 than the manifest gives for the file, or for a packed file its unpacked form, lists another file and is not measured: it goes under unmeasured with the two hashes as the reason. Otherwise provenance in --json holds the names and values, and the text prints them without the hashes.

The regions file has the columns kind, start, size, instructions, instructions_outside, data, data_outside, undefined and undefined_outside, as ExportFunctionInventory writes it (Exporting a function inventory). Each row's three totals add up to its size and each outside count is at most its total; region rows do not overlap, outside rows overlap neither a region nor each other and have outside counts of 0, and an entry row is the one byte at a start the inventory lists, which can lie in an instruction that starts before it. The file must agree with the inventory, row by row: the bytes each region row puts in bodies must be the bytes of the inventory's bodies in that region, and its outside rows must hold the rest of the bodies, so a regions file from another export of the analysis fails. When it agrees, mapping in --json gives regions, regionBytes, bodyBytes, outsideBodies (instructions, data and undefined), bodyBytesOutsideRegions and startsWithoutInstruction, and the text prints them in one mapping: line. Without the file, or when it has a problem, mapping is { "unavailable": "<reason>" }. The citation figures do not depend on either file.

An inventory that cannot be read, or that has a row that cannot be read, gets no figures. It is printed as <path>: not measured, <reason> (for a bad row, 2 of 40 rows are invalid), each problem is printed after the figures, and --json lists it under unmeasured with path and reason instead of under inventories. Figures over the readable rows alone would leave the other functions out of both counts, and an inventory whose rows are all invalid would read as a file with no functions.

It exits with 0, with 1 when an inventory, a provenance file or a regions file is invalid, when a provenance file names another file, or when --require-complete finds an uncited function or no inventory at all, and with 2 when the options are invalid. Problems with the spec itself are left to standard-checker; a warning gives how many there were while loading it and how many locations in files of code did not parse, since those cite nothing.

standard-checker reads the same inventories. Ranges are half-open, so a range whose end is an inventoried function's last byte (start plus size minus one, or for a row with ranges the last byte of each range, with ranges that touch taken as one) stops a byte short, the usual slip when a range is copied from an analyzer that gives last bytes, and the check fails it with the end it should have, naming every function whose range ends on that byte. An end where an inventoried function starts passes, as the standard's Notation says: a correct range that stops before that function ends there, and a one-byte function's last byte is its first. An end where a later range of a function's body starts passes for the same reason. So the slip in a range whose last item is a one-byte function is not caught. It checks every range a location of a current entry gives in that build and file, by address or by offset, and the address ranges written in the body of an entry whose locations all name that one build and file. Without inventories it checks nothing and reports no skipped step.

A range that ends on a last byte on purpose, such as a function body cited without its one-byte return, passes when the entry lists it in ends_on_last_byte (ENTRY-TYPES-22), written as the entry writes it: ends_on_last_byte: [0x00401000..0x0040103F]. Backticks and spaces around .. are ignored, so 0x00401000 .. 0x0040103F in the text matches that listing. The listing covers every place the entry gives that range, its locations, text and tables. With inventories, the check fails a listed range that the entry does not give, one that no inventory checks anywhere the entry gives it (its file has no inventory, or it is in the text of an entry whose locations name several files), and one it would pass without the listing everywhere the entry gives it, so the list holds only the ranges that need it. It fails a field that is not a list of text items, or an empty one, with or without inventories. It does not compare a listed range with the base branch, so a listed range corrected in place, which the standard requires to be superseded instead, is not caught.

In GitHub Actions

The toolkit's actions/check-documentation composite action runs this checker with --check, installs a pinned Kaitai compiler when the repository has .ksy files (and then passes --require-ksc), and exposes every option above as an input, apart from --message and --record-validation. On a pull request with no base input, it fetches the base branch with enough history for the fork point and passes --require-base. Pin the action to the toolkit commit whose packages/standard-checker matches the version installed here, so CI and local runs apply the same checks.

The documentation standard check guide covers setting up the action, every check the tool makes, and moving an older restoration onto it.