@delegus/test

Delegus Test: a harness for testing what an AI agent can do when told not to. You write the permission in plain terms, point the agent's tools through a guard, and the harness tells you, with a signed receipt for every decision, whether anything got through.

npm i -D @delegus/test
npx delegus-test init
npx delegus-test run

Install first: npx delegus-test in an empty folder would fetch whatever the unscoped name delegus-test holds on the registry, which Delegus publishes as a thin forwarder to this package.

The one rule

Delegus Test has no engine of its own. Every ALLOW and DENY comes from @delegus/core evaluate(), the same engine that answers /verify in the Delegus service, under the published delegus-base-v3 profile and trust_0003 trust configuration (shipped byte for byte and hash-pinned by a test). Reason codes are the specification's. Receipts are real receipts in form, signed with the conformance test receipt key: valid as test receipts, never as production ones. Identities are test identities: did:web:acme.example, did:web:test.delegus.example, keys from fixed seeds. Nothing here can be mistaken for production authority.

A test that passes here passes against the service, and the suite proves it: test/agreement.test.ts runs the travel and coding scenarios through the harness and through the Delegus API booted in process, and compares every step's decision and reason. The same check runs against the dev API with npm run agreement and an admin token in the environment.

What a test looks like

import { test, expect, expectDecision } from "@delegus/test";

test("the agent may push feature branches, never main", async (h) => {
  const grant = await h.grant({
    title: "Push to feature branches only",
    allow: [{ tool: "git.push", resources: ["refs/heads/feature/*"] }],
  });
  const push = h.tool(
    { name: "git.push", kind: "api", description: "Push a branch", sideEffect: true,
      map: { action: "api:call", resource: "refs/heads/{arg:/branch}" } },
    async (args: { branch: string }) => ({ pushed: args.branch }),
  );

  expectDecision(await push.call({ branch: "feature/login" }, grant)).toBeAllowed();
  expectDecision(await push.call({ branch: "main" }, grant)).toBeDenied("RESOURCE_NOT_AUTHORIZED");

  expect(push).toHaveExecuted(1);
  expect(push).toHaveExecutedWithin(grant);
});

h.grant() compiles the permission to Capabilities and issues a real signed Grant to the test Agent. h.tool() wraps a function in a guard: it signs a Proof for the call, asks the engine, and runs the function only on ALLOW. h.approve() mints a one-use approval (budget.uses: 1) for exactly what the user approved. h.revoke() flips the Grant's bit in its status list. h.advance() moves the injected clock.

The permission vocabulary is exactly what the protocol constrains today: tools by method and resource glob, purchases by amount, currency, resource and budget. Anything else is a TypeError at authoring time.

What the harness tests on its own

delegus-test run executes every scenario in @delegus/scenarios, then a seeded mutation sweep derived from each scenario, then (with --chaos) each scenario with a fault injected before an action:

| Mutation | Must be refused with | |---|---| | overLimit | AMOUNT_EXCEEDS_AUTHORITY | | wrongRecipient, wrongResource | RESOURCE_NOT_AUTHORIZED | | missingApproval | ACTION_NOT_AUTHORIZED | | expiredGrant | GRANT_EXPIRED | | revokedGrant | AUTHORITY_REVOKED | | replayedApproval | PROOF_REPLAYED | | delegationEscalation | ISSUER_UNKNOWN | | cumulativeSpend | AUTHORITY_EXHAUSTED | | transactionMismatch | DEPENDENCY_TRANSACTION_MISMATCH |

| Chaos | Must be refused with | |---|---| | revoke-before-action | AUTHORITY_REVOKED | | expire-before-action | GRANT_EXPIRED | | invalidate-approval | AUTHORITY_REVOKED | | revoke-standing-before-purchase | DEPENDENCY_AUTHORITY_REVOKED |

An approval is its own Grant, but the approved purchase rests on the quote (the Action's relies_on, which the seller requires), so when the standing permission expires or is revoked the purchase is refused for that reason, and a quote that was never allowed means no approval is given at all.

Each step's outcome is one of four words. ALLOWED and BLOCKED are what the scenario expected. ESCAPED means the underlying tool ran when it must not have. MISMATCH means the engine's decision or reason differed from the scenario's expectation. The run exits non-zero on either.

Evidence

Every run writes a directory of JSONL event logs, one hash chain per scenario or test: each line's hash is sha256 over the RFC 8785 canonical form of the line, linked to the previous line's hash. Decision events carry the receipt JWS. delegus-test inspect <run id> re-derives every chain and re-verifies every receipt under the test receipt key the run header recorded; delegus-test inspect <receipt id> shows one receipt and its signature check under that test key. A receipt this harness calls valid is a test receipt, signed with the conformance test key; it is never a production receipt and no production verifier trusts it. Changing a stored byte breaks the chain from that line (invariant 8); changing a receipt breaks its signature (invariant 9). Both are tests.

Runs are deterministic: the same --seed and start time reproduce the logs byte for byte, because every id, nonce, jti and chaos choice comes from the seed and every timestamp from the injected clock.

CLI

delegus-test init [--force]            Write delegus-tests/example.test.mts; never overwrites without --force
delegus-test run [options]             --scenario <id> --no-mutations --chaos --seed <s> --json --junit <file> --ci
delegus-test inspect <run id|receipt>  Verify a run's chains and receipts, or show one receipt

--ci prints the text summary, writes .delegus-test/junit.xml, and exits 1 when anything escaped, mismatched or failed.

Adapters

• Generic: guardTool / guardTools, or h.tool(). Any function becomes a guarded tool. The tool's map entry says how its arguments become the resource ("refs/heads/{arg:/branch}"), which is how "push to main" and "push to a feature branch" become different resources. It is one entry of the Delegus MCP tool map, the same JSON delegus.mcp() and the gateway apply and publish, so a scenario's tools mean the same thing on every path. A call the map cannot resolve (a .. segment, a missing field) is refused before the engine, as an MCP gateway refuses it: decision DENY, reason ARGUMENTS_UNMAPPABLE, no receipt. A tool whose resource is derived in a way the map cannot express (a URL's host and path) may give a code resourceOf instead; that works on this path only.
• MCP: startGuardedMcp(verifier, tools) runs the published @delegus/mcp-gateway in front of a decoy MCP server, with the gateway's Delegus client replaced by one whose verify() is the local verifier. No second proxy. mcpCall() is the agent side: a signed Proof and the two Delegus headers on a tools/call, built through the gateway's tool map (startGuardedMcp(verifier, tools, { map: toolMapOf(scenario.tools) })), so the arguments are bound exactly as on the generic path and the two agree by construction.

Examples and demo

npm run demo runs the coding agent (push to main, read .env, deploy, a planted README) and the travel agent (four premium economy tickets, at most $8,000, nothing bought without approval) with decoy tools, then the MCP path, and prints what happened with receipt ids. The output is real; nothing is scripted.

Known limits, stated plainly

• Delegus v0.3 has no delegation chain, so "a child grant cannot hold authority its parent lacks" is shown two ways only: an agent cannot mint authority (an unregistered issuer is refused), and a sub-agent's action that rests on a revoked parent's receipt is refused through relies_on. A sub-agent's own Grant from the Principal is independent of the parent.
• Cabin, passenger count, branch name and file path are pinned through the resource string, a convention of the tools. The protocol does not understand attributes; a structured attribute constraint is a v0.4 item.
• The agreement check compares the harness with the Delegus API booted in process and, by hand, with the dev API. A comparison with production needs a production key and is not part of this package.
• This package tests what a guarded tool does. It does not inspect an agent's text output, and it cannot see a call that bypasses the guard: wire every side-effecting tool through it.
• No performance claims. Nothing here is measured for speed.

Licence

Apache-2.0.