@tangle-network/agent-bench
v0.1.0
Published
The unified benchmark suite for agent-runtime agents: 18 adapters (commit0, enterpriseops-gym, trata-hedge, finsearchcomp, swe-bench, humaneval, …) behind one resolveAdapter registry, each with a real deterministic judge. Score any profile/skill/prompt ch
Downloads
170
Keywords
Readme
agent-runtime-bench
Private experiment workspace nested in agent-runtime; decoupled from its build/lint/release (the package builds src/, lints src tests examples — bench/ is none of those).
Read bench/HARNESS.md FIRST. It is the one maintained map: the commands, the rollout → corpus → selector → CI → gate data flow, the canonical-suite table, the wired/needs-creds/scaffolded matrix, and the gate one-liners — kept verified against source.
SWE-bench judge setup (the one block not in HARNESS.md)
python3 -m venv .venv && .venv/bin/pip install swebench # SWE-bench harness
pnpm install # tsx + link parent
# Docker daemon must be running (judges build/run per-instance images)The judge needs only Docker; workers need a model key (Tangle router TANGLE_API_KEY, or a direct provider).
