Agent Watchdog

Runtime loop and deadlock detection for multi-agent AI systems.

The problem

Multi-agent pipelines fail in ways that are hard to see from the outside:

• Argument loops — two agents debate the same point in different words, burning tokens indefinitely
• Silent deadlocks — agents wait on each other and never produce output
• Runaway iterations — a graph cycles forever because no exit condition was met

Existing tools like LangSmith and AgentOps show you what happened after the fact. Agent Watchdog intervenes at runtime — the same way a circuit breaker works in distributed systems.

Install

npm install agent-watchdog

Requires @langchain/langgraph and @langchain/core as peer dependencies.

Quick start

import { AgentWatchdog, createWatchdogRouter } from "agent-watchdog";
import { END } from "@langchain/langgraph";

const watchdog = new AgentWatchdog({
  maxIterations: 10,
  similarityThreshold: 0.55, // halt if recent messages are 55%+ similar to previous
  windowSize: 4,
});

// Drop into any LangGraph conditional edge
const router = createWatchdogRouter(watchdog, "agent_a");

graph.addConditionalEdges("agent_b", router, {
  agent_a: "agent_a",
  [END]: END,
});

Demo

Tool call loop

A research agent keeps rephrasing the same search query after every empty result. The message text changes each round, but the watchdog catches the near-identical tool arguments and halts before the iteration cap:

git clone https://github.com/your-handle/agent-watchdog
cd agent-watchdog && npm install
npm run example:tool-loop

Argument loop

Two AI agents debate whether AI is good or bad. The watchdog monitors the conversation and halts when they start recycling arguments.

ANTHROPIC_API_KEY=your_key npm run example:debate

Output:

============================================================
  AGENT WATCHDOG DEMO
  Topic: Is AI beneficial or harmful to humanity?
============================================================

[Optimist — round 1]
AI will drive tremendous progress in medicine, scientific discovery, and solving
complex problems like climate change. It's already accelerating drug development,
improving diagnostics, and automating dangerous work, freeing humans for more
creative pursuits. The risks require careful governance, but rejecting this
technology would mean abandoning solutions to humanity's greatest challenges.

[Pessimist — round 2]
AI systems pursuing misaligned objectives at scale could cause irreversible harm
faster than we can respond — we're building increasingly powerful tools whose
decision-making we don't fully understand. The promised benefits in medicine and
climate are speculative, while near-term harms are concrete: mass surveillance,
labor displacement, and weapons systems removing human judgment from lethal
decisions. Responsible AI governance isn't happening at the speed or depth needed.

[Optimist — round 3]
AI's benefits are already concrete and accelerating: real lives saved through
diagnostic AI, climate modeling improving energy efficiency now, and accessibility
tools enabling disabled individuals daily. The risks are real but solvable through
transparency, technical safety research, and international coordination.

... (continues for several rounds)

[Pessimist — round 8]
We've already created dependency that makes retreat impossible and failure
catastrophic simultaneously. We're now locked into deploying increasingly capable
systems we don't understand into domains where errors compound — financial systems,
power grids, military decisions.

============================================================
  AGENT WATCHDOG — INTERVENTION REPORT
============================================================
  Trigger    : loop_detected
  Iterations : 8
  Similarity : 71%  (threshold: 55%)
  Messages   : 9

  Last exchange:
    [Optimist] While AI accelerates scientific discovery, we must ensure equit...
    [Pessimist] The acceleration you describe comes at the cost of concentrating...
    [Optimist] Equitable access is precisely why open-source AI development mat...
    [Pessimist] Open-source development doesn't solve the compute access problem...

  Diagnosis : agents are recycling the same arguments.
  Action    : graph halted. No further tokens consumed.
============================================================

The watchdog correctly identifies that both agents are rephrasing the same core arguments and intervenes — stopping token consumption before the debate loops indefinitely.

API

new AgentWatchdog(config?)

| Option | Type | Default | Description | |--------|------|---------|-------------| | maxIterations | number | 10 | Hard cap on graph cycles | | similarityThreshold | number | 0.55 | Jaccard similarity above which an argument loop is declared | | toolCallThreshold | number | 0.7 | Jaccard similarity on tool args above which a tool-call loop is declared | | windowSize | number | 4 | Number of messages in each comparison window |

Methods:

watchdog.check(state)
// → "continue" | "loop_detected" | "tool_loop_detected" | "state_regression" | "limit_reached"
watchdog.similarity(state)       // → number (0–1), current argument similarity score
watchdog.report(state, decision) // → string, human-readable intervention report
watchdog.reportJson(state, decision) // → WatchdogReport, structured intervention report
watchdog.reset()                 // → clears seen-state history between independent runs

createWatchdogRouter(watchdog, nextNode, onIntervene?)

Returns a LangGraph conditional edge routing function.

• Routes to nextNode while the graph is healthy
• Routes to END and calls onIntervene(report) when the watchdog triggers
• If onIntervene is omitted, the report is printed to stdout

const router = createWatchdogRouter(watchdog, "agent_a", (report) => {
  myLogger.warn(report); // send to your logging system
});

How detection works

Four checks run on every cycle, cheapest first:

1. Iteration limit — hard cap regardless of content (limit_reached)
2. State regression — SHA-256 hash of the message window repeats a previous cycle (state_regression)
3. Argument loop — average pairwise Jaccard similarity between the recent and previous message windows exceeds similarityThreshold (loop_detected)
4. Tool call loop — Jaccard similarity on the arguments of the most-repeated tool call exceeds toolCallThreshold (tool_loop_detected)

recent   = last N messages (or tool calls)
previous = N before that
score    = Jaccard(recent, previous)

See ARCHITECTURE.md for the full breakdown of each check.

Langfuse integration

Report every similarity check — and any intervention — as a score on the run's Langfuse trace, so watchdog activity shows up next to your existing traces instead of only in stdout.

npm install @langfuse/client

import { randomUUID } from "crypto";
import { CallbackHandler } from "@langfuse/langchain";
import { LangfuseClient } from "@langfuse/client";
import { AgentWatchdog } from "agent-watchdog";
import { createLangfuseWatchdogRouter } from "agent-watchdog/langfuse";

const runId = randomUUID();
const langfuseHandler = new CallbackHandler();
const langfuse = new LangfuseClient();

const watchdog = new AgentWatchdog({ maxIterations: 10, similarityThreshold: 0.55 });
const router = createLangfuseWatchdogRouter(watchdog, "agent_a", { langfuse, traceId: runId });

graph.addConditionalEdges("agent_b", router, { agent_a: "agent_a", [END]: END });

await graph.invoke(input, { callbacks: [langfuseHandler], runId });

Important: traceId must be the same id passed as runId to .invoke() — that's what Langfuse's CallbackHandler uses as the trace id, and the router has no way to discover it on its own from inside a conditional edge. traceId also accepts a (state) => string function if you need to derive it per-run.

This adapter is a separate entry point (agent-watchdog/langfuse) with its own optional peer dependency on @langfuse/client — importing the base package (agent-watchdog) never pulls it in.

See examples/langfuse-debate.ts for a full runnable example (needs ANTHROPIC_API_KEY, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY).

Roadmap

• [ ] Human-in-the-loop pause (onPause callback, resume/abort)
• [ ] Token budget monitoring with early-warning callbacks
• [ ] Built-in webhook notifications (Slack, generic HTTP)
• [ ] OpenTelemetry exporter
• [ ] CrewAI and AutoGen adapters

See TODO.md for the detailed roadmap.

License

MIT