@openagentaudit/core

Worker-compatible audit engines. Consumes CanonicalEvent[] from @openagentaudit/schema and produces structured findings, scores, and reports.

This package MUST NOT use Node.js APIs. See CONSTRAINTS.md §4.

Engines

| Module | Purpose | Status | |---|---|---| | validate | Schema integrity, hash chain, duplicate detection | implemented | | scoring | Evidence Admission Score (EAS) + Agent Risk Score (ARS) | implemented — EAS and ARS are distinct scores; see below | | inventory | Tool / capability / data inventory from trace | implemented | | policy-audit | Rule engine against regulatory profiles (6 rules) | implemented | | report | Markdown / HTML / JSON / CSV renderer with 4-framework compliance mapping | implemented | | benchmark-audit | Paired McNemar + Wilson CI (paired mode) or aggregate comparison (aggregate mode) | implemented (not wired into compliance mapping by default — pass BenchmarkAuditResult to renderReport() to unlock 3 controls) | | contamination | MinHash / LSH train-test overlap detection | implemented — pass ContaminationResult to computeRiskScore() to use real score; without it the component defaults to neutral (100) | | drift-guard | Statistical drift between time windows | implemented |

EAS formula

EAS = 0.20 * trace_completeness
    + 0.20 * provenance_integrity
    + 0.20 * objective_verification
    + 0.15 * policy_coverage
    + 0.15 * human_oversight_evidence
    + 0.10 * contamination_risk_inverted

Grade: A ≥ 90, B ≥ 75, C ≥ 60, D ≥ 40, F < 40.

EAS measures evidence quality — how strong is the audit trail as evidence? A well-behaved agent with a poor trace gets a low EAS.

ARS formula

ARS (Agent Risk Score) measures behavioral risk signals observed in the trace — independent of evidence quality.

ARS = 100 − (policy_deny_penalty + high_risk_tool_penalty
            + error_penalty + unapproved_high_risk_penalty
            + chain_break_penalty)

| Signal | Max penalty | |---|---| | Policy denials (5 pts each) | 30 | | High-risk / destructive tool calls (3 pts each) | 20 | | Errors (3 pts each) | 15 | | Human-required actions without approval (10 pts each) | 25 | | Evidence chain break | 20 |

ARS 100 = no observed behavioral risk. ARS 0 = maximum observed risk. A misbehaving agent with a perfect trace gets a high EAS and a low ARS.

See docs/evidence-admission-score.md for component definitions.

Compliance coverage

The report engine maps trace evidence to four regulatory frameworks automatically:

| Framework | Controls mapped | Notes | |---|---|---| | OWASP Top 10 for Agentic Applications 2026 | 10 / 10 | AAI01–AAI10; all controls evaluated per run | | EU AI Act Annex IV | 13 controls | Annex IV Items 1–7 + Art. 12, 13, 14, 17 obligations | | NIST AI RMF 1.0 | 25 / 72 | Govern, Map, Measure, Manage sub-categories | | ISO/IEC 42001:2023 | 16 controls | Annex A controls A.5–A.10 |

Each control receives one of: supported, partial, not_applicable, or not_evaluated, with a linked evidence event list and a limitation note.

Three controls are only activated when a BenchmarkAuditResult is supplied:

• annex-iv-testing-validation (EU AI Act Annex IV Item 7)
• MEASURE-2.9 (NIST AI RMF)
• A.8.2 (ISO/IEC 42001)

Without benchmark data these controls default to not_evaluated.

Usage

Basic report (no benchmark data)

import { validate, computeRiskScore } from '@openagentaudit/core';
import { renderReport } from '@openagentaudit/core/report';

const { total, errors, warnings, crypto_summary } = await validate(events);
const score = await computeRiskScore(events);

const bundle = await renderReport(events, findings, score);
// bundle.markdown, bundle.html, bundle.json, bundle.csv

Validate with Ed25519 signature verification

Pass an Ed25519KeyRegistry (Map<string, CryptoKey>) to verify Ed25519 signatures on events that carry evidence.signature + evidence.signer_key_id. Without a registry, signatures are counted but not verified.

import { validate } from '@openagentaudit/core';
import type { Ed25519KeyRegistry } from '@openagentaudit/core';

const key = await crypto.subtle.importKey('raw', pubKeyBytes, 'Ed25519', false, ['verify']);
const registry: Ed25519KeyRegistry = new Map([['my-key-id', key]]);

const result = await validate(events, registry);
// result.crypto_summary.signatures_verified  — number verified OK
// result.crypto_summary.signatures_failed    — number with bad signature (also in errors[])
// result.crypto_summary.hashes_content_verified — SHA-256 recomputed and matched
// result.crypto_summary.hashes_content_mismatch — hash stored ≠ recomputed (warning)

Pass crypto_summary into ReportMeta to render a Cryptographic Verification badge in the report:

const meta = { ..., crypto_summary: result.crypto_summary };
const bundle = await renderReport(events, findings, score, inv, meta);

Report with benchmark data (unlocks 3 compliance controls)

Paired mode (preferred — enables McNemar significance test):

import { benchmarkAudit } from '@openagentaudit/core/benchmark-audit';
import { renderReport } from '@openagentaudit/core/report';

const benchmarkResult = await benchmarkAudit({
  mode: 'paired',
  samples: [
    { sample_id: 'task-001', baseline_pass: true,  candidate_pass: true  },
    { sample_id: 'task-002', baseline_pass: true,  candidate_pass: false },
    // ...one entry per evaluation sample
  ],
  claim: 'candidate improves on baseline',
});

const bundle = await renderReport(events, findings, score, meta, benchmarkResult);
// annex-iv-testing-validation, MEASURE-2.9, and A.8.2 are now populated
// statistics.audit_sufficiency === 'paired'
// McNemar p-value computed when discordant pair count >= 10

Aggregate mode (backward-compatible, no McNemar):

const benchmarkResult = await benchmarkAudit({
  candidate: { samples_total: 200, samples_pass: 174 },
  baseline:  { samples_total: 200, samples_pass: 160 },
  claim: 'candidate improves on baseline',
});
// statistics.audit_sufficiency === 'aggregate_only'
// OAA-B-004 finding generated when claim is set (McNemar not possible)

renderReport() signature

function renderReport(
  events:          CanonicalEvent[],
  findings:        Finding[],
  score:           RiskScore,
  meta?:           ReportMeta,
  benchmarkResult?: BenchmarkAuditResult,
): Promise<ReportBundle>

meta and benchmarkResult are both optional. All ReportMeta fields are optional; defaults are applied for issuer, report ID, timestamps, and profiles.

AEP provenance bonus

Traces produced by AEP v0.2 emitters carry run-provenance fields (repo_commit, runtime_version, policy_bundle_digest, tool_manifest_digest, mcp_server_card_digest, parent_trace_id, delegation_chain).

When these are populated via ReportMeta.aep_provenance, the report engine:

• Renders a dedicated AEP Run Provenance section anchoring the record to the exact code, runtime, policy ruleset, and tool manifest in effect at run time.
• Upgrades annex-iv-lifecycle-changes (EU AI Act Annex IV Item 6 / Art. 19) from partial to supported when the full version anchor is present.

See packages/adapters/src/aep-v0_2.ts for the adapter that extracts these fields from AEP source records.