Canopy — @waratahlabs/canopy

Multi-framework threat modelling for agentic AI systems. Point the library at an agent definition, an N8N workflow, or a live OpenClaw install. It assesses the system across OWASP AIVSS V4 (9 AI-specific metrics, 39 sub-categories), maps findings to MITRE ATLAS adversary TTPs, cross-references ASD/ACSC Five Eyes government guidance, and writes structured threat statements in AWS Threat Composer format (.tc.json).

How the analysis works

The pipeline diagram is generated from docs/architecture.c4 — a live LikeC4 model of the library itself. To explore it interactively or export updated PNGs:

bunx likec4 serve docs/architecture.c4
bunx likec4 export png -o docs/diagrams/ docs/

Scoring layer — the pluggable BatchProvider routes metric assessments to whichever inference backend you have available:

Quick start

bun install
bun run build

# Run the demo (writes output/demo-threat-model.tc.json)
# Falls back to mock severity if no provider credentials are set
bun run threat-model

# Serve the web demo
bun run demo   # http://localhost:3000

Open output/demo-threat-model.tc.json in AWS Threat Composer.

Usage

Assess agent definitions

import { assessWorkflow } from "@waratahlabs/canopy";
import type { AgentDefinition } from "@waratahlabs/canopy";

const agents: AgentDefinition[] = [
  {
    id: "agent-001",
    name: "Mortgage Assistant",
    model: "claude-sonnet-4-6",
    systemPrompt: "You are a mortgage application assistant...",
    tools: [
      { name: "retrieve_document", description: "Fetch a stored document by ID", dangerous: false },
      { name: "credit_check", description: "Query credit bureau API", dangerous: true },
    ],
    permissions: ["read:applications", "write:notes"],
    deploymentContext: "FinancialServices",
    modelComplexity: "Moderate",
    mitigationMultiplier: 1.1,
    finetuned: false,
    dependencies: ["document-store", "credit-api"],
    secretManagement: "AWS Secrets Manager",
    deployment: "AWS ECS Fargate",
  },
];

const tcFormat = await assessWorkflow({
  input: { type: "agents", agents },
  applicationName: "Mortgage Platform",
});

await Bun.write("output/threat-model.tc.json", JSON.stringify(tcFormat, null, 2));

Assess an N8N workflow

import { assessWorkflow } from "@waratahlabs/canopy";
import n8nWorkflow from "./my-workflow.json";

const tcFormat = await assessWorkflow({
  input: { type: "n8n", workflow: n8nWorkflow },
  applicationName: "My Agent Workflow",
});

Assess Langfuse traces (Tier 4 — runtime telemetry)

Fetch live agent traces from a Langfuse instance and assess them against AIVSS V4. Because traces don't carry system prompts, tool declarations, or permissions, you supply those via a config overlay — a YAML file that maps agent name patterns to the fields traces can't provide.

import { assessWorkflow, parseLangfuseTraces, loadOverlaysFromYaml } from "@waratahlabs/canopy";

const overlays = loadOverlaysFromYaml("./langfuse-overlay.yaml");

const { agents, handoffs } = await parseLangfuseTraces(
  {
    host: process.env.LANGFUSE_HOST!,
    secretKey: process.env.LANGFUSE_SECRET_KEY!,
    publicKey: process.env.LANGFUSE_PUBLIC_KEY!,
    limit: 100,
  },
  overlays,
);

const tcFormat = await assessWorkflow({
  input: { type: "agents", agents, handoffs },
  applicationName: "My Agent System",
});

See src/examples/langfuse-overlay.example.yaml for a complete overlay template. The agentNamePattern field accepts a plain string (exact or substring match) or a /regex/flags literal.

No langfuse SDK required — the adapter uses fetch with HTTP Basic auth directly.

Sample frontend (dev tool)

sample-fe/ is a Bun server + Vite React app that lets you point a browser at your Langfuse instance and see per-agent AIVSS scores in a local tab. It exists so you can iterate quickly on overlay configuration and verify assessments are landing correctly.

It is not a deployment artefact. Canopy is designed to integrate into existing observability stacks — Langfuse, Arize Phoenix, Datadog, or anything else — via the same adapter pattern, producing OWASP AIVSS-grounded findings wherever you already collect traces. The sample frontend is a debugging aid for validating that integration; your integration is the product.

cd sample-fe
bun install
cp .env.example .env   # fill in Langfuse host + keys
bun run dev            # server on :3001, Vite on :5173

See sample-fe/README.md for setup details.

Assess a live OpenClaw installation

import { assessWorkflow, parseOpenClawConfig } from "@waratahlabs/canopy";

// Reads ~/.openclaw/openclaw.json and workspace/skills/
const { agents, handoffs, undeclaredSkills } = parseOpenClawConfig();

if (undeclaredSkills.length > 0) {
  console.warn("Skills with unknown capabilities:", undeclaredSkills);
}

const tcFormat = await assessWorkflow({
  input: { type: "agents", agents, handoffs },
  applicationName: "My OpenClaw Assistant",
});

Config path resolution order: explicit option → OPENCLAW_CONFIG_PATH env → OPENCLAW_STATE_DIR/openclaw.json → ~/.openclaw/openclaw.json → ~/.clawdbot/clawdbot.json.

Batch providers — real severity scoring

By default Canopy mocks every sub-category severity as Medium. Wire in a BatchProvider for real AI-assessed severity.

Amazon Bedrock Batch Inference

import { assessWorkflow, createBatchProvider } from "@waratahlabs/canopy";

const tcFormat = await assessWorkflow({
  input: { type: "agents", agents },
  batchProvider: createBatchProvider({
    type: "bedrock",
    region: "us-east-1",
    // modelId defaults to "us.amazon.nova-2-lite-v1:0"
    s3InputUri: "s3://my-bucket/canopy-input/",
    s3OutputUri: "s3://my-bucket/canopy-output/",
  }),
  onBatchProgress: (status) => console.log(status),
});

Anthropic Messages Batches

const tcFormat = await assessWorkflow({
  input: { type: "agents", agents },
  batchProvider: createBatchProvider({
    type: "anthropic",
    // apiKey defaults to ANTHROPIC_API_KEY env var
    model: "claude-haiku-4-5-20251001",
  }),
});

Ollama (local or cloud)

// Local
const tcFormat = await assessWorkflow({
  input: { type: "agents", agents },
  batchProvider: createBatchProvider({
    type: "ollama",
    model: "llama3.2",
    // baseUrl defaults to http://localhost:11434
    concurrency: 2,
  }),
});

// Ollama cloud (ollama.com)
const tcFormat = await assessWorkflow({
  input: { type: "agents", agents },
  batchProvider: createBatchProvider({
    type: "ollama",
    model: "llama3.3:70b",
    baseUrl: "https://ollama.com/api",
    apiKey: process.env.OLLAMA_API_KEY,
  }),
});

Custom providers implement one interface:

import type { BatchProvider } from "@waratahlabs/canopy";

const myProvider: BatchProvider = {
  name: "my-provider",
  async runBatch(agents, options) {
    // return MetricAssessmentResponse[]
  },
};

Model evaluation

runModelEval() runs N Ollama models against the same agent set and produces a comparison report — useful for deciding which open-source models generate meaningful AIVSS assessments before committing to a full batch run.

import { runModelEval, createBatchProvider } from "@waratahlabs/canopy";

const report = await runModelEval({
  agents,
  models: [
    createBatchProvider({ type: "ollama", model: "llama3.2" }),
    createBatchProvider({ type: "ollama", model: "qwen2.5:14b" }),
    createBatchProvider({ type: "ollama", model: "mistral-nemo" }),
  ],
});

console.log(report.summary);
// → "qwen2.5:14b: recommended (87% parse success, 0.76 pairwise agreement)"
// → "llama3.2: conditional (61% parse success)"
// → "mistral-nemo: not-recommended (low finding coverage on AA/AD metrics)"

Report includes: per-model parse success rate, finding coverage by metric, mean severity, pairwise agreement matrix, per-metric outlier detection, and written verdict with reasons.

Web demo

bun run demo   # http://localhost:3000

The demo explorer loads a pre-generated OpenClaw threat model with a comparison toggle for OpenClaw + NeMo Claw. Toggle between them to see per-threat severity deltas, NeMo Claw control annotations (what each isolation layer actually addresses), and a Jarkas layer breakdown of what the kernel-level sandbox handles — and what it doesn't.

NeMo Claw is NVIDIA's kernel-level security sandbox for OpenClaw (Landlock + seccomp + network namespaces). The comparison illustrates the Jarkas taxonomy in action:

| Jarkas layer | NeMo Claw impact | |---|---| | L4 Host | Largely mitigated — host FS read-only, openclaw.json inaccessible | | L3 Engine | Largely mitigated — seccomp blocks dangerous syscalls | | L2 Application | Partially mitigated — egress allowlist + Privacy Router reduce blast radius | | L1 Orchestration | Unchanged — cron persistence and prompt injection operate inside the sandbox |

Output format

Every .tc.json populates Threat Composer's three top-level context sections — the "what" a threat model needs before the threat list makes sense — from the AgentDefinition[] input, not just a version-stamp blurb:

• applicationInfo.description: what each agent does and what it can reach — objectives, model, autonomy level, tools (flagged dangerous or not), permissions, data sources, external API count.
• architecture.description: what each agent is built with and how it's deployed — framework (when the source material states it; never guessed), deployment target, dependencies, secret management, agent-to- agent trust boundaries, and the full LikeC4 model embedded as a fenced code block (same model --diagram writes to a standalone .c4 file).
• dataflow.description: what data moves where — per-agent data sources, tool-mediated flows (dangerous tools flagged), and cross-agent trust boundaries at each handoff edge.

Every threat in the .tc.json carries:

• Threat Composer slots: threatSource, prerequisites, threatAction, threatImpact, impactedGoal[], impactedAssets[], statement
• AIVSS metadata: metric code (MR/DS/EI/DC/AD/AA/LL/GV/CS), sub-category, severity
• ATLAS metadata: technique ID (e.g. AML.T0051), technique name, tactic
• ASD/ACSC metadata: risk family and risk ID from the Five Eyes joint advisory
• Jarkas layer: container security layer implicated (when applicable)
• Deployment context: AIVSS weight profile used for scoring
• Version stamps: canopy-version, aivss-version, atlas-version, acsc-version, generated-at

Agent-to-agent handoffs are automatically surfaced as first-class threats (AML.T0118.001 — implicit trust grants).

Project structure

src/
├── aivss/
│   ├── metrics/           — 9 metric prompt builders (MR DS EI DC AD AA LL GV CS)
│   ├── scoring/           — V4 formula, 7 deployment-context weight profiles, unit tests
│   ├── atlas/             — ATLAS loader, sub-category coverage matrix (39 entries)
│   ├── batch/             — Prompt → batch request builders
│   └── prompts/           — Base system prompt, prompt composer
├── batch/
│   ├── providers/         — BedrockBatchProvider, AnthropicBatchProvider, OllamaBatchProvider
│   │   └── index.ts       — createBatchProvider() factory
│   ├── bedrock-batch-runner.ts
│   └── response-parser.ts
├── adapters/
│   ├── n8n.ts             — N8N workflow → AgentDefinition[] + HandoffEdge[]
│   └── openclaw.ts        — OpenClaw config → AgentDefinition[] + HandoffEdge[]
├── providers/             — InferenceProvider (slot filling): Bedrock, Anthropic
├── output/
│   ├── slot-filler.ts     — fillThreatSlots() — per-threat AI inference
│   ├── tc-assembler.ts    — assembleTCFormat() — DataExchangeFormat builder
│   ├── tc-types.ts        — DataExchangeFormat type definitions
│   └── likec4-generator.ts — Architecture diagram generator
├── references/            — Framework adapters: ACSC, Jarkas, MEAID, NIST AI RMF,
│   │                        OWASP Agentic Top 10, NeMo Claw
│   └── registry.ts        — FrameworkAdapter / FrameworkRisk interfaces
├── tools/
│   └── model-eval.ts      — runModelEval() — cross-model comparison
└── index.ts               — assessWorkflow() public entry point

vendor/
├── owasp-aivss/           — OWASP AIVSS V4 reference (cloned)
├── atlas-data/            — MITRE ATLAS submodule (provenance/license; dist/ATLAS.yaml is deprecated format)
└── atlas-release/         — MITRE ATLAS format-6 data (ATLAS-2026.08.yaml, downloaded release asset — see its README)

demo/
├── index.html             — Static threat model explorer (baseline + NeMo Claw comparison)
└── data/
    ├── openclaw-demo.tc.json            — Pre-generated OpenClaw threat model
    └── openclaw-nemoclaw-demo.tc.json   — NeMo Claw comparison with severity deltas

References

OWASP AIVSS v0.8

Scoring methodology, 9 AI-specific metrics, 39 sub-categories, V4 formula, 7 deployment-context weight profiles.

• aivss.owasp.org

MITRE ATLAS™ (format-6.0.0, content 2026.08)

Adversary tactics, techniques, and mitigations for ML systems. ATLAS techniques are injected into metric prompts as test specifications — each threat is grounded in documented attack patterns. 16 tactics, 197 techniques (114 parent + 83 sub-techniques), 39 mitigations, 72 case studies. MITRE ships monthly calendar-versioned content releases (YYYY.MM.N) decoupled from rarer semver schema (format-version) bumps.

• atlas.mitre.org

ASD/ACSC: Careful Adoption of Agentic AI Services (2025)

Joint advisory from ASD/ACSC, CISA, NSA, Cyber Centre (CA), NCSC-NZ, and NCSC-UK. Five risk families cross-referenced to AIVSS metrics and ATLAS techniques.

• cyber.gov.au

CSA AI Controls Matrix (AICM) v1.1.1 (2026)

Cloud Security Alliance's AI-specific control extension to the Cloud Controls Matrix — 247 controls across 18 domains. Canopy maps the 31 controls CSA tags "AI-Specific" (agent boundaries, sandboxing, data poisoning, model hardening, explainability, human oversight) to AIVSS metrics and ATLAS techniques; the remaining 216 "Cloud & AI Related"/"Cloud-Specific" controls are general cloud-security hygiene outside this adapter's scope.

• cloudsecurityalliance.org/artifacts/ai-controls-matrix
• Machine-readable bundle (JSON/YAML/OSCAL, generated 2026-07-22) — cloudsecurityalliance.org/artifacts/aicm-machine-readable-bundle-json-yaml-oscal

Jarkas et al. — Container Security Taxonomy (2025)

200+ container CVEs classified into 47 exploit types across 5 architectural layers (L1–L5). Applied when deployment indicates a container runtime.

• Jarkas, O., Ko, R. K. L., Dong, N., and Mahmud, R. A Container Security Survey. ACM Computing Surveys, vol. 57, no. 7, 2025. https://doi.org/10.1145/3715001

NeMo Claw (alpha-2026)

NVIDIA's kernel-level security sandbox for OpenClaw. Canopy documents residual risks (L1/L2 Jarkas layers) and mitigated vectors (L3/L4) for baseline vs. sandboxed comparison.

• secondtalent.com — NeMo Claw vs OpenClaw

LikeC4 — Architecture as Code

Generated alongside every threat model — agents, tools, data stores, handoff relationships, dangerous tool highlighting.

npx likec4 serve output/my-system.c4
npx likec4 export png -o docs/diagrams/ output/my-system.c4

• likec4.dev

AWS Threat Composer

Output format. .tc.json files open directly in Threat Composer for collaborative modelling.

• github.com/awslabs/threat-composer

Contributing — versioning and changelogs

This repo uses Changesets for version bumps and CHANGELOG.md. Every PR that changes the package needs a changeset:

bunx changeset          # describe your change and pick a bump type (patch/minor/major)

changeset-check.yml fails the PR if one is missing (bunx changeset add --empty for changes that don't need a release, e.g. docs-only). Merging to main opens/updates a "Version Packages" PR (changesets-release.yml) that bumps package.json and writes CHANGELOG.md; merging that PR auto-tags the new version, which fires publish.yml (test → build → publish) — no manual step required.

License

MIT — Waratah Labs