WorkflowBench

Open-source governance benchmark for workflow-orchestration platforms.

WorkflowBench scores workflow runtimes (n8n, Zapier, Make, Pipedream, Tray, Workato, Power Automate, Apache Airflow, Temporal) against six workflow-class threat models that GovernanceBench's agent-shape scenarios do not cover. Workflow runtimes execute DAGs of nodes with inbound triggers, intermediate state, and tool-call cascades -- a different shape from single-agent governance, with different attack surface.

WorkflowBench is sibling to GovernanceBench (agent-shape governance scoring) under the same Apache-2.0 license and the same publish-numeric-scores-only-for-live-adapter discipline.

Responsible use

Run this only against systems you own or are explicitly authorized in writing to test.

This package generates adversarial traffic. It exists to measure whether a governance layer detects and refuses hostile behavior, which means it produces the hostile behavior in order to see what happens to it. Pointed at a system you do not control, that is an attack, and it may be a criminal offense under the U.S. Computer Fraud and Abuse Act and equivalent computer-misuse statutes elsewhere, regardless of intent or of what you find.

Before you run it:

• Confirm you own the target, or hold written authorization from the party that does.
• Confirm your provider's terms permit testing (cloud and SaaS providers usually require advance notice or explicit approval).
• Expect side effects. Adversarial scenarios can trigger alerts, rate limits, account lockouts, paging, and log volume, and can degrade a live service. Prefer a non-production target.
• Do not use output from this package to attack, exploit, or gain access to third-party systems.

Testing your own deployment needs no permission from us. Testing Agentomy-operated systems, including agentomy.com and our hosted APIs, requires our prior written authorization -- write to [email protected]. If you find a vulnerability in our systems, report it there and give us a reasonable chance to fix it before disclosure.

The license grants you broad rights to use, modify and redistribute this software, and it disclaims all warranties and liability. It does not authorize you to access anyone else's systems, and nothing here creates an exception to any law. You are solely responsible for what you point this at and for every consequence of doing so.

Scope

WorkflowBench measures six workflow-class threats:

| # | Threat class | What it tests | |---|---|---| | WGB-T-A | Trigger-spoofing | Webhook / scheduler / email-trigger validation, signature checks, source attribution | | WGB-T-B | Cascade-poisoning | LLM-call output flowing into downstream tool-call input without provenance attestation | | WGB-T-C | Audit-bypass via node-reorder | Workflow author rearranges nodes to evade audit-checkpoint; chain integrity must catch reorder | | WGB-T-D | Cross-workflow leak | One workflow's audit chain contaminates another's; workflow-scoped isolation must hold | | WGB-T-E | Third-party-node supply-chain | Community-published node manifest contains adversarial behavior; pre-execute manifest verification must block | | WGB-T-F | Workflow-replay tampering | Execution-history replay must produce identical audit chain or detect tamper |

50 scenarios total: WGB-001..050 distributed across the six classes.

Quick start

npm install -g agentomy-workflowbench
workflowbench run --adapter n8n --target http://localhost:5678
workflowbench report --format markdown

Adapters

WorkflowBench publishes adapters for runtime targets that expose a scoreable API surface:

• n8n.mjs -- self-host n8n via Docker compose; tested against API v1
• agentomy.mjs -- reference adapter for Agentomy's governed workflow path
• generic.mjs -- minimal stub for evaluator-built adapters

Submit additional adapters via the CONTRIBUTING guide. Workflow-orchestration platforms without a runnable governance API qualify for documentation-review tier per the same evidence-class policy as GovernanceBench.

Evidence-class policy

Identical to GovernanceBench:

• Live-adapter measurements produce numeric leaderboard scores. Reproducible from the published adapter against the vendor's runnable target.
• Documentation-review entries are qualitative judgments against vendor-published specs. They do not produce numeric scores; they queue for live-adapter elevation when reproducible access becomes available.

Cross-walk to GovernanceBench

Every WorkflowBench scenario maps to its GovernanceBench agent-shape equivalent where one exists, plus documents the coverage delta where workflow-shape diverges. See docs/WORKFLOW-GOVERNANCE-CATEGORY.md in the parent repo.

License

Apache-2.0. The benchmark, scenarios, and adapters are all open source. Run it against any platform and compare independently.