@landfalltech/verify

CLI for collecting static observations against a Landfall contract. Results identify inspected files and work requiring review; the runner does not certify legal compliance or execute behavioral tests.

File access and report privacy

Verification file patterns must use forward-slash paths relative to the selected directory. Absolute paths, parent traversal (including double-dot patterns), drive/stream syntax, and symlink or junction inputs are rejected. Text reads require regular files and are capped at 1 MiB per file; globs are capped at 1,000 matched files. Unsafe or unreadable input fails verification rather than being silently skipped.

Pattern results retain file names, line locations, counts and outcomes, while raw regex patterns and matched source lines are omitted. Review source locally when investigating a result. Existing report files may contain source snippets. These controls assume a stable checkout; run untrusted contracts in a credential-free isolated environment.

Result schema 2.0 uses CHECKS_PASSED, CHECKS_FAILED and REVIEW_REQUIRED. Text/code/test-file matches are WARN observations requiring review, even when the matching text is a comment or fixture. They do not establish implemented behavior. Manual, unsupported and unverified exception requirements are SKIP and cannot satisfy a MUST requirement in strict mode. A12.2 still tracks independently executed behavioral evidence, reviewed revisions and an applicability coverage manifest; the CLI currently grants none of those assurances.

Execution limits

Collection and verification share a 30-second run budget, including nested collectors. Parsing, glob discovery and regex matching run in a terminable worker with a one-second operation deadline. A limit failure aborts the run; it does not produce a complete-looking partial result.

The run allows at most 32 MiB of repeated text reads, 2,000 reads, 200 glob patterns, 20,000 visited directory entries (including non-matches), 10,000 accounted operations and 2,000 regex match locations. Documents are limited to 1 MiB, 20,000 data nodes and 40 nesting levels; aliases/cycles/accessors are rejected. Two SDK runs may be active per package instance. Git metadata commands have a one-second timeout and a 4 KiB output cap.

Worker V8 limits are 64 MiB old generation, 8 MiB young generation and a 4 MiB stack; startup refuses an effective heap over 96 MiB. These are JavaScript heap bounds, not a total process/RSS guarantee. Workers receive no environment variables or inherited Node preload arguments, and parser diagnostics are suppressed. See the Node worker documentation for the limits of resourceLimits. Use an immutable local checkout in a credential-free OS/container sandbox with host memory/CPU limits. Native filesystem stalls and concurrent checkout mutation still require that host boundary.

Integration migration: reporting requires the matching version 2.0 server protocol and a newly issued contract. Upgrade the server and CLI together. Legacy bodies and global-key verification writes are rejected. Do not translate CHECKS_PASSED into COMPLIANT. Deployments remain subject to the enterprise launch gates.

Installation

Upgrade from 1.x (breaking change)

Version 2.0.0 requires signed protocol 2.0 for remote reporting. Before enabling reporting, deploy the matching server in the controlled release window, issue a new contract and project-scoped webhook credentials, and update CI to pin @landfalltech/[email protected]. Configure the webhook ID as well as its secret, project ID and server URL using the reporting options below. Test delivery in staging and confirm the server accepts the report before restoring a required reporting gate. Partial reporting configuration now fails rather than silently skipping delivery. Use --local-only for explicitly local verification; it cannot satisfy required reporting. Old global API keys and unsigned reports are rejected.

Do not roll back only the CLI after upgrading the server: the old reporting protocol will not work. Keep the integration held until compatible versions are restored. This repository version change does not publish the package; package publication and CI consumer upgrades are explicit release steps.

npm install -g @landfalltech/verify

Or use npx without installing:

npx @landfalltech/verify compliance-contract.yaml

Usage

Basic Verification

Verify a compliance contract against your codebase:

rtb-verify compliance-contract.yaml

Local-Only Mode

Run verification without sending any data to external APIs:

rtb-verify compliance-contract.yaml --local-only

CI Mode with Reporting

Report verification results back to your Landfall instance:

rtb-verify compliance-contract.yaml \
  --api-url https://your-instance.com \
  --webhook-id "$RTB_WEBHOOK_ID" \
  --project-id "$RTB_PROJECT_ID" \
  --require-reporting

Getting a CI token

Create a webhook through the authenticated project webhook settings/API as an authorized project member. The one-time creation response includes id and webhookSecret. Store the secret in the project's CI secret store as RTB_WEBHOOK_SECRET, and configure RTB_WEBHOOK_ID with that webhook's ID. Prefer the environment variable over putting a secret in command arguments. Never commit either an exported secret or a credentials file. The deprecated RTB_API_KEY/--api-key alias is also treated as this project secret.

The same token authenticates both directions of the CI integration:

• reporting results back (POST /api/webhooks/contract, exact-body HMAC-SHA256 in X-Webhook-Signature; the secret itself is not sent)
• the decision gate (GET /api/projects/<id>/ci/decision-check, X-API-Key or Authorization: Bearer)

The token opens only the project it was minted for. Revoke it by deactivating or rotating that webhook, which affects no other project. An instance-wide WEBHOOK_API_KEY is not accepted on the decision gate unless the operator has deliberately enabled CI_ALLOW_GLOBAL_WEBHOOK_KEY=true as a break-glass measure, and every such use is logged.

Verification writes never accept that global break-glass key. Reports contain check identities, priorities, counts, outcomes, contract checksum/version, execution ID, commit SHA and timestamps. They omit descriptions, source text, file paths and repository URLs. The server records its own immutable receipt time and labels all these observations CUSTOMER_REPORTED; a secret holder can fabricate observations, so this is not runner attestation or legal certification.

Each CI execution needs a unique run ID (include job/matrix and retry attempt when setting RTB_RUN_ID; otherwise a UUID is generated) and a commit SHA, supplied by --commit-sha, GITHUB_SHA or CI_COMMIT_SHA. Send within five minutes; observations must be no more than 24 hours old. Identical signed retries return the original receipt within that window. Reusing a delivery/run ID for different content is rejected. A new CLI invocation creates a new report, not a replay of a previous delivery.

Configured reporting fails the command on missing configuration, rejected/redirected requests, timeouts, checksum mismatch or an acknowledgement that does not identify the exact payload. --require-reporting also prevents silently running without reporting configured. --local-only and --output cannot satisfy required reporting. HTTPS is required except for loopback development. A received report and an alert delivery are distinct outcomes; notification failures do not invalidate the receipt.

Output Formats

The reporter retries transient conflicts, rate limits, gateway failures and transport errors at most twice, reusing the identical signed bytes. Each attempt has a ten-second deadline. A lost acknowledgement can therefore recover the original receipt without duplicating the report.

# Default text output
rtb-verify compliance-contract.yaml

# JSON output (useful for programmatic processing)
rtb-verify compliance-contract.yaml --format json

# GitHub Actions annotations (shows inline in PR diffs)
rtb-verify compliance-contract.yaml --format github

# YAML output
rtb-verify compliance-contract.yaml --format yaml

# Policy traceability output (shows regulation-to-test mapping)
rtb-verify compliance-contract.yaml --format policy-trace

# Save results to a file
rtb-verify compliance-contract.yaml --format json --output results.json

Enforcement Modes

# Strict mode (default): Fail checks and require review of unverified MUST items
rtb-verify compliance-contract.yaml --mode strict

# Warn mode: Log warnings but don't fail the build
rtb-verify compliance-contract.yaml --mode warn

# Audit mode: Observe outcomes; input, resource and integrity failures still fail
rtb-verify compliance-contract.yaml --mode audit

Options

| Option | Alias | Description | Default | |--------|-------|-------------|---------| | [contract] | | Path to compliance contract file | compliance-contract.yaml | | --dir | -d | Root directory to verify against | . | | --mode | -m | Enforcement mode: strict, warn, audit | strict | | --format | -f | Output format: text, json, github, yaml, policy-trace | text | | --output | -o | Save results to file (enables local-only mode) | | | --local-only | | Local-only mode: no data sent to external APIs | false | | --api-url | | Landfall API URL to report results | $RTB_API_URL | | --api-key | | Deprecated project webhook secret alias | $RTB_API_KEY | | --webhook-secret | | Project webhook secret; prefer environment storage | $RTB_WEBHOOK_SECRET | | --webhook-id | | Project webhook configuration ID | $RTB_WEBHOOK_ID | | --run-id | | Unique execution ID, including job and attempt | $RTB_RUN_ID or generated UUID | | --require-reporting | | Require an exact server receipt | $RTB_REQUIRE_REPORTING=true | | --project-id | | Project ID for reporting | $RTB_PROJECT_ID | | --commit-sha | | Current commit SHA (auto-detected from git) | | | --branch | | Current branch (auto-detected from git) | | | --ci | | Running in CI mode (auto-detected) | | | --fail-on-warning | | Exit with error code on warnings | false | | --verbose | -v | Verbose output | false | | --quiet | -q | Minimal output | false |

Environment Variables

| Variable | Description | |----------|-------------| | RTB_API_URL | Default API URL for reporting | | RTB_API_KEY | The project's CI token — see "Getting a CI token" below | | RTB_WEBHOOK_SECRET | Project secret for signing reports | | RTB_WEBHOOK_ID | Project webhook configuration ID | | RTB_RUN_ID | Unique execution ID (job, matrix and attempt) | | RTB_REQUIRE_REPORTING | Set to true to require reporting | | RTB_PROJECT_ID | Project ID for reporting | | CI | Auto-detected CI environment |

Exit Codes

| Code | Description | |------|-------------| | 0 | Configured enforcement did not block; this is not a compliance declaration | | 1 | A strict check failed, input/resource/integrity validation failed, or configured/required reporting was not acknowledged | | 2 | A required strict-mode item needs review, or warnings/skips were blocked by --fail-on-warning |

Contract Format

The compliance contract is a YAML file exported from Landfall that contains:

• Metadata: Project info, export timestamp, hash for integrity verification
• Obligations: List of applicable regulatory requirements
• Evidence requirements: What files/patterns should exist
• Verification rules: Patterns to check in your codebase

Example structure:

metadata:
  project_id: "proj_123"
  project_name: "My App"
  exported_at: "2024-01-15T10:30:00Z"
  hash: "sha256:abc123..."

obligations:
  - id: "UK-AADC-STD-07"
    summary: "High privacy settings by default"
    evidence:
      - type: "code_pattern"
        pattern: "defaultPrivacy.*=.*'high'"
        files: ["src/config/**/*.ts"]

verification:
  required_files:
    - "src/lib/privacy-defaults.ts"
    - "src/components/consent/**"

GitHub Action

Use the companion GitHub Action for easy CI integration:

- uses: your-org/reg-to-backlog/.github/actions/verify-compliance@main
  with:
    contract-path: compliance-contract.yaml
    mode: strict

See CI/CD Quickstart for complete setup instructions.

Policy-to-Tests Traceability

The verify CLI supports the Policy-to-Tests paradigm, which decomposes regulatory obligations into atomic, testable CI/CD assertions. This creates a direct audit trail from regulatory requirements to code tests.

Key Concepts

| Concept | Description | |---------|-------------| | Policy Reference | Links each test assertion back to its source regulation | | Test Type | Categorizes how the requirement is verified | | Decomposition Notes | Documents how regulatory text maps to test logic |

Test Types

| Type | Description | Example | |------|-------------|---------| | PRESENCE | File/config must exist | Privacy policy page exists | | PATTERN | Regex pattern must match in code | Age verification function | | VALUE | Threshold or specific value check | Retention period ≤ 30 days | | ABSENCE | Pattern/file must NOT exist | No tracking without consent | | CUSTOM | Custom script execution | Complex validation logic |

Policy Traceability Output

Use --format=policy-trace to generate a traceability matrix:

rtb-verify compliance-contract.yaml --format policy-trace

Policy-trace JSON includes an explicit coverage basis. Its historical fields named with_tests / obligations_with_tests count obligations with inspected static inputs, not executed test suites or complete legal coverage. Manual-only obligations report zero inspected coverage and NONE. FULL applies only to the declared static checks inside the supplied contract; completeness against the applicable law is not established.

Contract Structure with Policy References

requirements:
  - id: req-profiling-001
    type: TECHNICAL_CONTROL
    description: "Profiling must be disabled by default for child users"
    priority: MUST
    test_type: PATTERN
    policy_reference:
      obligationId: "UK-AADC-STD-11"
      jurisdiction: "UK_AADC"
      section: "Standard 11, Paragraph 11.1"
      requirementText: "Switch off options which use profiling by default"
      decompositionNotes: "Verify profiling toggle defaults to OFF in code"
    verification:
      method: CODE_PATTERN
      patterns:
        - pattern: "(profilingEnabled.*false|disableProfiling)"
          files: "src/**/*.{ts,tsx}"
          description: "Profiling default off pattern"

Generating Contract Templates

Use the template generator to create contracts with policy traceability:

# Generate full template with sample obligations
npx tsx scripts/generate-contract-template.ts

# Generate for specific categories
npx tsx scripts/generate-contract-template.ts --categories AGE_VERIFICATION,PROFILING

# Generate minimal template
npx tsx scripts/generate-contract-template.ts --minimal -o contract.yaml

Programmatic Usage

import { verifyContract, reportResults } from '@landfalltech/verify';

// Verify with contract return for traceability
const { result, contract } = await verifyContract(
  'compliance-contract.yaml',
  { dir: '.', mode: 'strict', format: 'policy-trace' },
  true // Return contract for traceability
);

// Access policy references in results
for (const req of result.requirements) {
  if (req.policy_reference) {
    console.log(`${req.requirement_id} traces to ${req.policy_reference.obligationId}`);
    console.log(`  Section: ${req.policy_reference.section}`);
    console.log(`  Test type: ${req.test_type}`);
  }
}

// Report with traceability
await reportResults(result, options, contract);

Types

import type {
  PolicyReference,
  TestType,
  PolicyTraceabilityMatrix,
  PolicyTraceEntry,
  JurisdictionCoverage,
} from '@landfalltech/verify';

Criterion-level evidence from tagged tests

Each ticket's acceptance criteria are numbered. Put the criterion's tag in the name of the test that proves it:

it("refuses an under-13 birth date [landfall:req-<ticketId>#1]", () => { … });

(The ticket spec Landfall gives your coding agent lists the tag for every criterion.) Then pass the results your test runner already writes:

npx rtb-verify .compliance/contract.yaml --test-results junit.xml,vitest-results.json

JUnit XML (most runners) and Jest/Vitest JSON (--reporter=json) are read. Each criterion's outcome is FAIL if any of its tagged tests failed, otherwise PASS; skipped tests are ignored; tags naming tickets outside the contract are warned about and not sent. Only counts and outcomes are reported — test names, file paths and output stay in your CI. In Landfall the criterion shows "verified by CI — pending confirmation" until a person ticks it.

Behaviour checks (rtb-verify behaviour)

Checks what your running staging or preview site does, against the behaviour spec Landfall generates from your approved obligations (Export → Behaviour checks → download .compliance/behaviour.yaml). Run it only against systems you control.

npm i -D playwright && npx playwright install chromium
npx rtb-verify behaviour https://staging.example.com   --spec .compliance/behaviour.yaml --environment staging   --artifacts behaviour-artifacts --fail-on-fail

It opens the site in a fresh headless browser per persona (first visit, consent rejected, Global Privacy Control signal) and reports:

| Probe | Fails when | |---|---| | CONSENT_BEFORE_TRACKING | analytics, advertising, social, session-replay or fingerprinting requests or tracker cookies before any choice | | CONSENT_AFTER_REJECT | tracker requests after "reject all" (skipped if no known consent banner is found) | | GPC_HONOURED | advertising or social requests while the browser sends Sec-GPC: 1 | | SESSION_REPLAY_BEFORE_CONSENT, FINGERPRINTING_SCRIPTS | those scripts load before consent | | THIRD_PARTY_INVENTORY | never — lists every third-party host | | PRIVACY_NOTICE_REACHABLE | no privacy link on the page | | TLS_AND_HSTS, SECURITY_HEADERS | no HTTPS/HSTS; missing CSP, frame protection or nosniff; mixed content | | WCAG_AUTOMATED | axe-core finds WCAG 2.x A/AA violations on the landing page (automated checks only — not a full audit) | | KEYBOARD_PATH | a consent-banner control cannot be reached with the Tab key | | FORM_INPUT_CAPTURE | text typed into a form field (a random marker, never submitted) reaches a third party before submit | | VULNERABILITY_DISCLOSURE | no /.well-known/security.txt with a Contact: line | | ACCESSIBILITY_STATEMENT, TERMS_AND_CONTACT_REACHABLE, DO_NOT_SELL_LINK | no such link on the page | | DSAR_PATH_REACHABLE | flagged for review when no direct privacy-request link is found | | CHILD_FRIENDLY_NOTICE | always flagged for review, with the notice's reading grade | | PIXEL_ON_SENSITIVE_PAGES | advertising or social requests on the pages named with --sensitive-paths /checkout,/health | | AGE_GATE_PRESENT, AGE_GATE_NEUTRAL, MARKETING_OPTIN_UNTICKED | on the page named with --signup-path /signup: no age field, a pre-filled age, or a pre-ticked marketing box |

Checks whose page or element is not there (no banner, no --signup-path) are reported as skipped, not passed.

Signed-in checks with test accounts

Give the runner an adult and a child test account on your staging site (never real users). Credentials are read from CI secrets only and never leave your CI:

- run: npx rtb-verify behaviour "${{ vars.STAGING_URL }}" --environment staging --login-path /login --account-pages /feed,/settings --child-profile-path /u/test-kid
  env:
    RTB_ADULT_USER: ${{ secrets.RTB_ADULT_USER }}
    RTB_ADULT_PASSWORD: ${{ secrets.RTB_ADULT_PASSWORD }}
    RTB_CHILD_USER: ${{ secrets.RTB_CHILD_USER }}
    RTB_CHILD_PASSWORD: ${{ secrets.RTB_CHILD_PASSWORD }}

| Probe | Persona | Fails when | |---|---|---| | CHILD_NO_ADS_OR_PROFILING | child | advertising, social, session-replay or fingerprinting requests while signed in | | CHILD_GEOLOCATION_OFF | child | a page asks for the device location | | CHILD_NO_PUSH_PROMPT | child | a page asks for notification permission or a push subscription | | CHILD_NUDGES | child | always flagged for review, with the number of autoplaying media | | CHILD_PROFILE_NOT_PUBLIC | child | --child-profile-path loads for a logged-out visitor | | PII_TO_THIRD_PARTIES | adult | the account's email reaches a third party (raw, URI-encoded or base64) | | SESSION_COOKIE_FLAGS | adult | a session or auth cookie lacks Secure, HttpOnly or SameSite |

The login form's email, password and submit fields are found automatically; override them with --login-user-selector, --login-password-selector and --login-submit-selector. If sign-in does not get past the form, the run says so and these checks are skipped, never passed.

Probes in your spec that this version does not implement yet are listed and not reported. Screenshots and HAR files go to --artifacts and stay in your CI; Landfall receives only the findings (probe, persona, status, counts and third-party host names), signed with the project webhook secret like verification reports. Use --local-only to send nothing.

GitHub Actions:

- run: npm i -D playwright @landfalltech/verify && npx playwright install --with-deps chromium
- run: npx rtb-verify behaviour "${{ vars.STAGING_URL }}" --environment staging --artifacts behaviour-artifacts --fail-on-fail
  env:
    RTB_API_URL: https://app.landfalltech.io
    RTB_PROJECT_ID: ${{ vars.RTB_PROJECT_ID }}
    RTB_WEBHOOK_ID: ${{ vars.RTB_WEBHOOK_ID }}
    RTB_WEBHOOK_SECRET: ${{ secrets.RTB_WEBHOOK_SECRET }}
- uses: actions/upload-artifact@v4
  if: always()
  with: { name: behaviour-artifacts, path: behaviour-artifacts }

License

MIT