@animakit/debate

Multi-agent debate with a number attached: Φ̂ measures whether your agents actually influenced each other — or just talked past each other.

Why this exists

Multi-agent frameworks (CrewAI, AutoGen) make agents "talk", but nothing guarantees the talk integrated anything. You usually get N independent answers stapled together, or one loud agent drowning out the rest — and no way to tell which.

debate runs an N-agent debate with two things nobody else ships:

1. Mandatory cross-pollination. Every agent states an independent position (Round 1a), then sees the others' positions and revises (Round 1b). That revision creates a causal dependency between agents — the thing that makes a debate a debate instead of a panel of monologues.
2. A measurable integration score, Φ̂. A single number in [0,1] that says how much the agents actually moved after seeing each other. Φ̂ ≈ 0 → the debate was theater. Φ̂ > 0.3 → real integration.

"CrewAI agents debate. These agents debate and you can measure whether the debate changed anything."

The final answer is produced by a Nash-equilibrium synthesis prompt: each agent carries an explicit utility function, and the synthesizer finds the point where no agent would concede further without making the whole answer worse. Tensions are shown, not averaged away.

Zero runtime dependencies. Bring your own LLM (a 3-line adapter — see below). Extracted from a production agent.

Does the extra round actually help? (the ablation)

The whole premise rests on one claim: the extra cross-pollination round produces measurably better answers than the "independent agents + synthesis" shape everyone else ships. That is the launch gate, and it is measured honestly — not asserted.

| Mode | Calls | Blind-judge win rate | |---|---|---| | (a) no Round 1b — independent + synthesis (typical CrewAI/AutoGen shape) | N+1 | pending production run | | (b) full protocol — 1a → 1b → synthesis | 2N+1 | pending production run |

Honest status: the ablation harness ships in benchmarks/ablation.ts and is ready to run with any real LLM + a blind judge over ≥30 questions. It has not been run in this repo (no API keys here) and no numbers are invented. If mode (b) doesn't beat mode (a) by enough to justify the extra N calls, the protocol gets revised before launch. See SPEC.md §6.

Quickstart

npm install @animakit/debate
# or: pnpm add @animakit/debate

import { runDebate, presets } from '@animakit/debate';

// A 3-line LLM adapter — this one is Anthropic; any SDK works (see below).
const llm = async (system: string, user: string) => {
  const r = await anthropic.messages.create({
    model: 'claude-haiku-4', max_tokens: 1024,
    system, messages: [{ role: 'user', content: user }],
  });
  return { text: r.content[0].type === 'text' ? r.content[0].text : '' };
};

const result = await runDebate('Should we raise our prices?', presets.starterTrio, {
  debateLLM: llm,
});

console.log(result.synthesis); // the Nash-equilibrium answer
console.log(result.phi_hat);   // e.g. 0.41 — the agents genuinely moved

presets.starterTrio is a generic optimist / skeptic / pragmatist. Swap in your own participants when you're ready (see Utility functions).

The protocol

Four rounds. Rounds 0 is optional; 1a/1b/2 are the core.

Round 0  (optional)   Audience modeling
                      Infer the recipient's belief / what they're seeking /
                      receptivity, so the answer lands with the right framing.

Round 1a              Independent positions        ── N parallel calls
                      Each agent states its core position (~60 words),
                      WITHOUT seeing the others. This is the pre-integration
                      baseline used to compute Φ̂.

Round 1b              Cross-pollination            ── N parallel calls
                      Each agent now sees the others' positions and rewrites
                      its answer. Agent X's Round-1b answer depends causally
                      on agent Y's Round-1a position → the system is no longer
                      decomposable into independent parts → Φ̂ > 0.

Round 2               Nash-equilibrium synthesis   ── 1 call
                      The synthesizer resolves the (now cross-informed)
                      perspectives into one answer, showing real tensions
                      instead of averaging them.

Cost: 2N+2 LLM calls for N participants (+1 if audience modeling is on). Latency: round0 + max(round1a) + max(round1b) + synthesis — the parallel rounds fan out with Promise.allSettled.

Debates that survive partial failure

Both parallel rounds use Promise.allSettled, so a flaky provider doesn't sink the whole debate:

• an agent that fails in 1b falls back to its 1a position;
• if only one agent survives, it's returned directly (no pointless synthesis);
• if every agent fails in 1a, runDebate throws.

roundFailurePolicy — degrade vs strict

Default is degrade (v0.1.0 behavior). Set strict when a partial panel is unacceptable — e.g. compliance reviews where every specialist must contribute:

await runDebate(question, panel, {
  debateLLM: llm,
  roundFailurePolicy: 'strict', // throws if ANY participant fails in 1a or 1b
});

Sub-debate failures always degrade (fall back to the requesting agent's stripped 1b position) — never throw.

v0.2.0 — protocol events & recursive sub-debates

Three mechanisms ported from Engram's production Mesa de Decisión (NestJS SaaS) into this zero-dependency package. Mechanisms only — Engram's domain panels, mesa routing, and RAG integration stay in the commercial product.

onEvent — typed protocol lifecycle

Stream the full debate protocol through a callback (replaces wiring Engram's SSE union yourself):

await runDebate(question, participants, {
  debateLLM: llm,
  onEvent: (e) => {
    switch (e.type) {
      case 'phase_start':       /* round0 | round1a | round1b | synthesis */ break;
      case 'agent_position':    /* name, round, text */ break;
      case 'agent_failed':      /* name, round */ break;
      case 'sub_debate_start':  /* domain, requestedBy, participants */ break;
      case 'sub_debate_complete': /* domain, synthesis */ break;
      case 'sub_debate_failed': break;
      case 'phi_update':        /* phi_hat */ break;
      case 'synthesis_complete': break;
      case 'debate_complete':   /* durationMs */ break;
    }
  },
});

Ordering guarantees: phase_start precedes that phase's agent_position events; all 1a/1b positions precede phi_update; phi_update precedes synthesis (phase_start → synthesis_complete); debate_complete is always last. Parallel 1a/1b positions may arrive in any order among agents. Nested sub-debate events carry depth (0 = root).

onLog / onMetrics are unchanged. The package still never writes to stdout.

Recursive sub-debates

When a participant needs a specialist panel mid-debate, it emits a language-neutral marker on its own line in Round 1b:

REQUEST_SUB_DEBATE: security

Configure domain → panel mapping; the engine runs a nested runDebate, feeds the synthesis back, and re-issues that participant's 1b position:

await runDebate(question, mainPanel, {
  debateLLM: llm,
  subDebates: {
    domains: {
      security: [secSpecialistA, secSpecialistB],
      legal: [contractSpecialist],
    },
    maxDepth: 1,      // default — root + one nested level (Engram's effective depth)
    maxPerDebate: 1,  // default — one sub-debate per parent debate
    trigger: defaultSubDebateTrigger, // or your own parser
  },
});

The 1b prompt includes bilingual instructions only when subDebates.domains is non-empty and depth allows it. On sub-debate failure, the requesting agent keeps its stripped 1b position (degrade, never throw).

interpretPhiHat

Map Φ̂ to a qualitative band — defaults calibrated on production Engram distributions (high ≥ 0.70, medium ≥ 0.50), documented as heuristic:

import { interpretPhiHat } from '@animakit/debate';

interpretPhiHat(0.72);                        // 'high'
interpretPhiHat(0.55);                        // 'medium'
interpretPhiHat(0.3, { high: 0.8, medium: 0.6 }); // 'low'

Why Φ̂

Most of the time you want a single, honest signal: did the debate actually integrate anything, or did the agents just restate themselves? Φ̂ is that signal.

Concretely, Φ̂ is the mean Jaccard divergence between each agent's Round-1a position and its Round-1b perspective:

Φ̂ = mean_agent( 1 − |words(1a) ∩ words(1b)| / |words(1a) ∪ words(1b)| )

• Φ̂ → 0: agents barely changed their wording after seeing each other — low integration.
• Φ̂ → 1: agents rewrote substantially in response to each other — high integration.

For the curious: this is a cheap, transparent proxy for integrated information (Φ) from Integrated Information Theory — the property that a system's whole carries information its independent parts don't. We don't claim it is Φ (computing true Φ is intractable); it's a word-overlap stand-in that's O(tokens) and needs no model call. computePhiHat() is exported standalone so you can score any pre/post pair of texts.

import { computePhiHat } from '@animakit/debate';

computePhiHat({ a: 'raise prices to protect margin' },
              { a: 'raise prices to protect margin' });        // 0   — nothing changed
computePhiHat({ a: 'raise prices to protect margin' },
              { a: 'hold prices, cut costs instead' });        // ~1  — completely revised

Utility functions

Giving each agent an explicit utility function ({ maximizes, penalizes }) is what turns a synthesis from a bland average into a real equilibrium: the synthesizer can see which agent is trading off what, and resolve it deliberately.

presets.animaProduction is a real example — the exact 5 specialists that run in a production agent, with their real utility functions (domain specifics anonymized, game-theoretic structure intact):

| Participant | maximizes | penalizes | |---|---|---| | CEO | long-term survival, growth, strategic coherence | existential risk, future-compromising decisions | | Scout | complete, verified information; current context | decisions on incomplete data, blind spots | | Architect | technical quality, scalability, zero tech debt | shortcuts, rushed calls, over-engineering | | Legal | compliance, zero regulatory exposure | unmitigated legal/tax risk, ambiguity | | Sales | probability of closing, decision speed, deal value | commercial friction, deals lost to over-caution |

import { runDebate, presets } from '@animakit/debate';

const result = await runDebate(
  'A big client wants a feature that breaks our data model. Do we build it?',
  presets.animaProduction,
  { debateLLM: cheapModel, synthesizerLLM: strongModel }, // cheap rounds, strong synthesis
);

Or define your own — a participant is just { name, systemPrompt, utility? }:

const participants = [
  { name: 'Growth', systemPrompt: 'You optimize for acquisition velocity.',
    utility: { maximizes: 'new users and activation', penalizes: 'anything that slows signup' } },
  { name: 'Trust',  systemPrompt: 'You optimize for user safety and retention.',
    utility: { maximizes: 'long-term trust and retention', penalizes: 'dark patterns, churn risk' } },
];

Heterogeneous debate

Any participant can carry its own llm, so different models with different strengths debate each other (e.g. an execution model vs. an adversarial-review model). Default is the global debateLLM.

await runDebate(question, [
  { name: 'Builder', systemPrompt: '…', llm: fastModel },
  { name: 'RedTeam', systemPrompt: '…', llm: adversarialModel },
], { debateLLM: fastModel, synthesizerLLM: strongModel });

API reference

runDebate(question, participants, config): Promise<DebateResult>

| config field | type | default | notes | |---|---|---|---| | debateLLM | LLMComplete | — | required. Used for the debate rounds. | | synthesizerLLM | LLMComplete | debateLLM | Model for the final synthesis. | | audienceModeling | { enabled, audienceContext? } | off | Round 0: model the recipient. | | wordLimits | { position?, integrated?, synthesis? } | 60 / 150 / 380 | per-round word budgets. | | synthesisSections | ('tensions'\|'equilibrium'\|'answer')[] \| string | all three | a raw string is used verbatim as the structure. | | language | 'en' \| 'es' | 'en' | language of the internal prompts. | | onMetrics | (m) => void | — | receives { phi_hat, agentsInvolved, durationMs }. | | onLog | (level, message, data?) => void | silent | structured logs; the package never writes to stdout itself. | | onEvent | (e: DebateEvent) => void | — | typed protocol lifecycle events (v0.2.0). | | subDebates | { domains, maxDepth?, maxPerDebate?, trigger? } | — | recursive specialist panels (v0.2.0). | | roundFailurePolicy | 'degrade' \| 'strict' | 'degrade' | participant failure handling in 1a/1b (v0.2.0). |

Returns DebateResult: { synthesis, positionStatements, perspectives, audienceModel, agentsInvolved, durationMs, phi_hat }.

interpretPhiHat(phi, bands?): 'low' | 'medium' | 'high'

Heuristic Φ̂ bands. Defaults: high ≥ 0.70, medium ≥ 0.50 (Engram production calibration).

defaultSubDebateTrigger(text): string | null / stripSubDebateMarker(text): string

Parse and clean the REQUEST_SUB_DEBATE: <domain> marker.

computePhiHat(positions, integrated): number

Unicode-aware standalone Φ̂ over two Record<string, string> maps keyed by the same names. Returns [0,1].

tokenize(text): Set<string>

The word tokenizer behind Φ̂. Unicode letter classes, min word length 3.

presets

presets.animaProduction (5 specialists) and presets.starterTrio (optimist/skeptic/pragmatist).

LLM adapters

LLMComplete = (systemPrompt, userPrompt) => Promise<{ text: string }>. There is no LLM SDK anywhere in this package — you supply the function, so any provider works.

// Anthropic
const anthropicLLM = async (system, user) => {
  const r = await anthropic.messages.create({
    model: 'claude-haiku-4', max_tokens: 1024,
    system, messages: [{ role: 'user', content: user }],
  });
  return { text: r.content[0].type === 'text' ? r.content[0].text : '' };
};

// Vercel AI SDK
import { generateText } from 'ai';
const vercelLLM = async (system, user) => {
  const { text } = await generateText({ model: myModel, system, prompt: user });
  return { text };
};

// OpenAI
const openaiLLM = async (system, user) => {
  const r = await openai.chat.completions.create({
    model: 'gpt-5.1-mini',
    messages: [{ role: 'system', content: system }, { role: 'user', content: user }],
  });
  return { text: r.choices[0]?.message.content ?? '' };
};

// Ollama (local, $0)
const ollamaLLM = async (system, user) => {
  const r = await fetch('http://localhost:11434/api/chat', {
    method: 'POST',
    body: JSON.stringify({ model: 'gemma3', stream: false,
      messages: [{ role: 'system', content: system }, { role: 'user', content: user }] }),
  });
  const j = await r.json();
  return { text: j.message?.content ?? '' };
};

Retries, rate-limiting, and timeouts belong in your adapter — the package deliberately doesn't own them.

Benchmarks

Every claim here is reproducible from benchmarks/. We separate what's measured from what's pending real production data:

| Benchmark | What it reports | Status | |---|---|---| | latency.ts | Protocol overhead — orchestration cost with an instant mock LLM (not model latency). Gate: p99 < 1ms. | ✅ measured — run pnpm bench | | phi-distribution.ts | Distribution of Φ̂ over real production debates (SPEC Table 1). | ⏳ pending production data export (n≈434) — no numbers fabricated | | ablation.ts | Cross-pollination ablation, the launch gate (SPEC Table 2). | ⏳ pending — ready to run with an injected real LLM + judge |

The latency benchmark isolates the part this package is responsible for; real debates are dominated by the 2N+2 model calls. The Φ̂-distribution and ablation numbers require a real LLM and the production dataset, so they are intentionally left un-run rather than faked.

What this is NOT

• Not routing/activation — deciding when to debate (which agents to wake up) is the router's job (@animakit/neuromorphic-router). This package only runs the debate you hand it.
• Not an agent framework — there's no Agent class; a participant is { name, systemPrompt, utility? }.
• Not persistence — onMetrics hands you Φ̂; storing it is yours.
• Not retry/rate-limiting — that lives in your LLMComplete.

Part of ANIMA

Agentic Neuro-Inspired Memory Architecture — battle-tested cognitive architecture for LLM agents, extracted from 53 sprints in production. Siblings: @animakit/complexity-scorer, @animakit/homeostasis, @animakit/git-guardrails.

License

MIT © Justine Serna