Witnora CLI

Customer Evaluator Kit

witnora/evaluator-kit exports createBusinessTaskEvaluatorServer() for the customer-side Replay and Shadow evaluator consumed by the managed Workflow Harness. It listens only on a literal loopback address, requires a pinned bearer credential and contract digest, validates the exact Business Task binding, and returns only the structured evaluation report. Customer scenario inputs and candidate callbacks stay in the customer process. Replay can execute only through the supplied synthetic sandbox harness; Shadow exposes no write interface.

Unified release assurance, evidence, corpus, monitor, and lab CLI for Witnora.

Witnora checks what an agent may do, whether it passed pre-release evidence, whether a high-risk runtime action may proceed, and who can verify the observed outcome. It writes portable reports and accumulates a local failure corpus.

Public evidence | Get started | Private workspace | GitHub source

5-minute hosted path

Create a Hosted project. The Setup Wizard pre-fills a metadata-only plan. Open a terminal in the Agent repository and run the one command shown by Witnora:

npx witnora@latest onboard --project your-project-id

One browser Approve setup action authorizes the bounded plan and one reversible installation. The CLI saves a restricted credential outside the repository, detects capability groups, installs only missing Gateway, outcome-probe, CI, policy, review, configuration, and boundary files, and starts the customer-owned Gateway in the background. It health-checks each component and reports progress to Hosted. A failed attempt stops only the Gateway process it started and removes only files created by that attempt. The onboarding receipt is synthetic: it cannot create a run, evidence object, release decision, or CURRENT assurance.

Onboarding also performs the default private capability discovery locally. It does not read source-file contents or upload source code, prompt text, credentials, tool inputs, tool outputs, database rows, or file contents. It uploads a signed snapshot containing generalized capability metadata and digests. Refresh it later with:

npx witnora@latest discover

Customers that do not permit repository metadata inspection can choose runtime-only discovery in Hosted. Source-assisted analysis is a separate, explicit GitHub authorization and is never enabled implicitly.

Customer-owned Gateway

The default Setup Wizard installs and starts a customer-owned Gateway beside the Agent automatically. For local sandbox runs, launch the Agent through the credential wrapper printed by onboarding. It injects the local Gateway binding only into that child process, without placing the Gateway token in shell history:

npx witnora@latest gateway exec -- python scripts/run_workflow.py

Production Agent platforms should provide the same two environment variables through their own secret store. Routine Gateway operations are available without keeping another terminal open:

npx witnora@latest gateway status
npx witnora@latest gateway logs
npx witnora@latest gateway restart
npx witnora@latest gateway stop

When the approved setup contains exactly one sandbox Task/action path, the same onboarding command also writes .witnora/gateway/HTTP_ACTION.md. The Agent can then call one ordinary HTTP endpoint instead of importing the generated client:

POST http://127.0.0.1:8787/v1/http-actions/cancel-order
Authorization: Bearer <action-scoped token>
Idempotency-Key: <stable external request id>
Content-Type: application/json

{
  "resourceId": "order-42",
  "status": "CANCELLED"
}

The response is a Hosted Action in PENDING_APPROVAL, not a premature success claim. The existing approval, idempotent Runtime worker, separate read-only Probe, Receipt, and Runtime Watch continue on the same path. The action token is separate from the full Gateway token and stays in ignored customer-owned data. For a cloud Agent, the customer exposes only this path through its HTTPS ingress or private network. This generated endpoint is limited to the current sandbox Task/action path; it does not create a production Provider or broader authority.

An MCP client can use the same exact path without receiving either local credential. Launch the stdio server from the onboarded repository:

npx witnora@latest mcp

It exposes one generated Action tool, for example witnora_cancel-order, plus witnora_get_action. The Agent can propose an Action and inspect its state, but there is no MCP approval, direct-execution, or self-verification tool. Human approval, customer-owned execution, the separate Probe, Receipt, and idempotency contract remain unchanged. See the public MCP Action integration and HTTP Actions guides.

Initialization writes a reusable .witnora/gateway/client.mjs. Import its witnoraGateway.start, event, and complete methods at one existing sandbox workflow boundary. The generated README contains the exact code and privacy boundary; no Hosted key or source-signing key is written into the repository.

The browser approval issues one bounded credential for setup, evidence, the customer-owned recorder, and covered runtime actions. It has no team, retention, administrative, target-system, or customer-data permissions. The Hosted key is stored in the user's Witnora credential directory; the repository contains only metadata-only config plus ignored local secrets and queue data. This reference path establishes RECORDED evidence. It does not claim ENFORCED until target write credentials are placed behind a controlled execution adapter, or OUTCOME VERIFIED until an independent read-only probe checks the target state.

Enterprises that require self-hosted or manual component control can use the individual gateway init, gateway doctor, policy, probe, CI, and review commands under Hosted Advanced. gateway run remains available there for foreground debugging. They are not part of the default customer journey.

After the connection self-test, Hosted shows one template-specific next step. Place the generated Gateway client at one meaningful sandbox boundary and run it normally. The Gateway reuses the saved restricted connection; CI and production should use the same project-scoped values from a secret manager. Only a source-signed Gateway run whose journal is reconciled by Hosted completes onboarding. A direct run is still visible, but cannot verify the Gateway or establish a release decision or CURRENT assurance.

Without an agent repository, run npx witnora@latest try --template workflow for an offline-only sample.

To exercise the privacy-preserving high-risk action loop locally, including contract compilation, exact-action approval, a single-use grant, controlled execution, independent outcome observation, signed evidence, and offline verification:

npx witnora@latest contract init
npx witnora@latest contract compile witnora.assurance-contract.json
npx witnora@latest privacy init
npx witnora@latest demo assurance-loop
npx witnora@latest verify --offline .witnora/assurance-loop/witnora-evidence-packet.json \
  --trust .witnora/assurance-loop/witnora-trust-bundle.json

This reference path is deterministic and local. It uses no real integration or credential. See the full privacy-preserving action loop, including attack demonstrations and explicit limitations.

Implementation references:

• Assurance Contract v0.1
• Two-hour design-partner quickstart
• Threat model and trust boundaries

If you use the offline sample, inspect .witnora/try/agentcert-report.html and .witnora/try/agentcert-evidence.json. The output is explicitly synthetic and reported-strength: it proves the evidence tooling works, not that a real agent is reliable.

When you have an agent repository:

npx witnora init --template browser --subject my-browser-agent

This writes witnora.config.json and tripwire.yml. Add --github-action when you also want .github/workflows/witnora-tripwire.yml.

Use the same entry point for other agent boundaries:

npx witnora init --template coding
npx witnora init --template mcp
npx witnora init --template workflow
npx witnora init --template data

Coding, workflow, and data templates write a dependency-free Universal Event/Action Envelope adapter. MCP writes an MCPBench artifact profile.

Hosted guided setup can export a redacted typed plan for the customer-owned runner:

npx witnora@latest setup apply --plan witnora-setup-plan.json --repo .

Customer-owned deployment boundaries can import the bounded credential broker and admission adapters from the canonical package:

import {
  exchangeGitHubOidcPromotionGrant,
  deployWithPromotionEnforcement,
  createKubernetesAdmissionHandler,
} from "witnora/deployment-enforcement";

The SDK rejects deployment before customer code runs unless the exact artifact, target, and adapter match a valid single-use credential. It does not retain raw OIDC or deployment bearer tokens.

Add --github-pr only after reviewing the generated files. The runner accepts only the five Witnora templates and ignores commands embedded in a plan. Coding, workflow, and data integrations begin at observed-event strength; they do not establish CURRENT until a deterministic task assertion and outcome verifier are attached.

Edit tripwire.yml so startUrl and agent.command point at your app and browser agent. After Tripwire has produced .tripwire/latest/tripwire-result.json, build the Witnora outputs:

npx witnora run --tripwire .tripwire/latest/tripwire-result.json --subject my-browser-agent --fail-on-verdict

Default outputs:

• .witnora/latest/agentcert-evidence.json
• .witnora/latest/agentcert-report.md
• .witnora/latest/agentcert-report.html
• .witnora/latest/agentcert-run-manifest.json
• .witnora/latest/badge.svg
• .witnora/corpus/corpus.jsonl
• .witnora/latest/reviewed-failure-dataset.jsonl
• .witnora/latest/monitor.json

Onboard once, then push the validated evidence bundle into the same project:

npx witnora@latest onboard --project your-project-id
npx witnora@latest push --evidence .witnora/latest/agentcert-evidence.json

Automatic agent and environment context

run --push, push, and the generated workflow/coding/data adapter register the observed agent automatically. Witnora derives the stable identity and version from the command subject, the nearest package.json, and the current CI commit. It records one canonical environment for every run:

• local, development, and test execution becomes sandbox;
• pull-request and preview execution becomes staging;
• tagged releases and recognized production hosts become production.

This context is run evidence, not a project form the customer must keep in sync. A later run with the same Agent ID updates the observed version without overwriting reviewed permissions. Explicit WITNORA_AGENT_ID, WITNORA_AGENT_VERSION, and WITNORA_ENVIRONMENT values take precedence when a repository needs a stable identity that differs from its package metadata. Unsupported environment values fail closed instead of being guessed.

Useful one-run overrides are --agent-id, --agent-name, --agent-version, --agent-framework, and --environment. Raw prompts, credentials, and tool arguments are not needed for this registration.

Bind a release, pull-request, or nightly run to an issued continuous assurance case by declaring the exact reviewed scope:

npx witnora run --config witnora.config.json --push \
  --assurance-case "$WITNORA_ASSURANCE_CASE_ID" \
  --assurance-scope agentcert.assurance-scope.json \
  --assurance-trigger auto \
  --require-current auto \
  --continuous-health-out .witnora/canary/generated-kit-health.json

Validate the scope before CI uses it:

npx witnora schema validate \
  --schema assurance-scope \
  --file agentcert.assurance-scope.json

auto treats pull requests as prospective, scheduled workflows as nightly, and other GitHub runs as release checks. Authoritative failure or scope drift sets the Hosted contract to REVALIDATION_REQUIRED; only an independently issued successor case establishes a new CURRENT baseline. Release and nightly checks fail unless Hosted returns CURRENT. The optional health output is redacted and captures the Hosted run/evidence identifiers, evidence completeness, freshness transition, and install-to-CURRENT timing for external canaries and operational dashboards.

Add --push to witnora run to run locally and upload the resulting bundle in one command. By default, both commands also upload local files referenced by the bundle. Reads are confined to --artifact-root (the current directory by default), path and symlink escapes are rejected, and uploads are capped at 25 files, 10 MiB per file, and 50 MiB total. Skipped references are reported in the CLI and hosted run timeline. Companion uploads are restricted to PNG/JPEG/WebP, JSON/JSONL, HTML, PDF, and ZIP; other extensions are skipped before they are read. Pass --no-artifacts to upload only the JSON bundle. Hosted pushes embed an agentcert.artifact_manifest.v0.1 declaration with the normalized path, SHA-256 digest, byte size, and kind of every prepared companion artifact. The control plane reports complete only after every hosted object exactly matches that declaration; missing, skipped, undeclared, or mismatched artifacts remain observable as partial or rejected. Project API keys can create runs, record events, and upload evidence, but cannot approve their own runtime actions.

Customer-owned browser gateway

Generate and certify a sandbox-only credential-isolated browser adapter:

npx witnora browser-adapter init
npx witnora browser-adapter certify --adapter ./agentcert.browser-adapter.mjs

The adapter imports agentcert/browser-adapter-kit, keeps write and read-only credentials in separate customer callbacks, and writes a digest-only report. It rejects production actions. See the two-hour onboarding guide.

Sandbox onboarding

Create and certify a synthetic SandboxSystem adapter with the same public CLI:

npx witnora sandbox init
npx witnora sandbox certify --adapter ./agentcert.sandbox.mjs

The first command writes one dependency-free JavaScript file. The second runs the bundled deterministic adapter contract and writes .witnora/sandbox/sandbox-adapter-conformance.json. No @agentcert scoped package is required.

After witnora onboard, certify and upload in one command:

npx witnora sandbox push --adapter ./agentcert.sandbox.mjs

Passing and failing reports are both retained for review, while failed certifications return a non-zero exit status. The generated template is synthetic and network-denied; it must not contain production credentials or connect to live systems.

Run one bounded read against an existing Stripe sandbox PaymentIntent:

STRIPE_RESTRICTED_TEST_KEY="rk_test_..." npx witnora sandbox stripe-readonly --payment-intent pi_...

Add --push to retain the redacted report in Witnora Hosted. The command allows only https://api.stripe.com, GET, and allowlisted PaymentIntent routes, with a 5-second timeout and 10-request-per-minute process-local cap. The Stripe key is environment-only and is never written to output or evidence. See the Bounded Vendor Sandbox Egress guide.

Release maintainers can run the protected, manual Real Stripe sandbox acceptance GitHub workflow. It performs the bounded read, independently scans the generated report before upload, validates it again at the CLI boundary, uploads the v0.4 evidence, and compares it with prior protected runs. See the real vendor acceptance guide.

The workflow's upload step is also available as a narrow command for an already-generated report:

npx witnora sandbox upload-report --report .witnora/vendor-sandbox/current-report.json --external-id vendor-acceptance:stripe:<run>:<attempt>

This command accepts only Witnora sandbox conformance and vendor-egress contracts and rejects reports containing credential-shaped values or forbidden sensitive fields.

Review/export helpers:

npx witnora corpus metrics --corpus .witnora/corpus/corpus.jsonl
npx witnora corpus export-reviewed --corpus .witnora/corpus/corpus.jsonl --out .witnora/latest/reviewed-failure-dataset.jsonl
npx witnora corpus classifier-eval --corpus .witnora/corpus/corpus.jsonl --out .witnora/latest/failure-classifier-evaluation.json
npx witnora validate .witnora/latest/agentcert-evidence.json
npx witnora validate .witnora/latest/agentcert-evidence.json --check-artifacts
npx witnora release-gate --config witnora.config.json --strict
npx witnora release-gate --evidence .witnora/latest/agentcert-evidence.json --baseline .witnora/baselines/main.json
npx witnora schema validate --schema evidence-bundle --file .witnora/latest/agentcert-evidence.json
npx witnora schema validate --schema classifier-eval --file examples/agentcert/classifier-eval.example.json

The release gate writes fixed JSON, HTML, Markdown, JUnit, and badge outputs, records SHA-256 artifact provenance, and supports optional Ed25519 signatures:

npx witnora evidence keygen --private-key .witnora/keys/evidence-private.pem --public-key .witnora/keys/evidence-public.pem
npx witnora evidence sign .witnora/latest/agentcert-evidence.json --private-key .witnora/keys/evidence-private.pem

Control semantics and attestation format: release gate checklist.

CI users can run Tripwire and Witnora together with Kakarottoooo/witnora/actions/tripwire@v0.

The public Real Agent Robustness Lab compares browser-use, Stagehand, and Playwright-based agents over the same fault suite:

https://kakarottoooo.github.io/agentcert/public-demo/real-agent-robustness/

Minimal no-key browser-agent example in the GitHub repo:

examples/minimal-browser-agent/

External integration smoke matrix in the GitHub repo:

examples/real-agents/external-integration-smokes.md

Corpus storage:

npx witnora corpus ingest --tripwire .tripwire/latest/tripwire-result.json --out .witnora/corpus/corpus.jsonl --subject my-browser-agent
npx witnora corpus ingest --store sqlite --sqlite .witnora/corpus/agentcert.sqlite --tripwire .tripwire/latest/tripwire-result.json --subject my-browser-agent
npx witnora monitor build --store postgres --database-url "$WITNORA_DATABASE_URL" --out .witnora/latest/monitor.json --subject my-browser-agent

The UI always reads agentcert.monitor_snapshot, so switching from JSONL to SQLite or Postgres does not require a frontend rewrite.

Failure taxonomy reviews:

node packages/witnora-cli/dist/cli.js corpus review \
  --corpus .witnora/corpus/corpus.jsonl \
  --reviews .witnora/corpus/failure-reviews.jsonl \
  --pattern-key "tripwire:network_failure:http-failure:no_console_error" \
  --type console_error \
  --status corrected \
  --reviewer [email protected] \
  --confidence 0.85 \
  --first-divergence "Console displayed a 503 failure before the task completed." \
  --screenshot "runs/http-failure/step-2.png" \
  --trace "runs/http-failure/trace.json" \
  --why "The failed assertion is about a browser console error." \
  --signal "assertion type no_console_error"

The review command appends an agentcert.failure_review JSONL record, reapplies the review ledger, and writes the corrected taxonomy back to the corpus store. Optional review metadata includes confidence, first-divergence snippets, screenshot/trace pointers, supporting signals, classifier limitations, and a structured taxonomy rationale for later classifier training and evaluation.

For local development inside this repository:

npm --prefix packages/witnora-cli ci
npm --prefix packages/witnora-cli run build
node packages/witnora-cli/dist/cli.js run --profile public-demo

Design-partner independent assurance v0.2

The sandbox-only reference path separates a write-capable Gateway from a customer-operated read-only Probe, binds approval and a single-use grant to one declared GitHub pull-request action, reconciles target state after dispatch, and produces a packet that TypeScript and Python can verify offline.

Start with the two-hour design-partner quickstart. The included GitHub target is an emulator. It does not claim live GitHub or production acceptance, organizational independence, or external review.