@songforgeai/agent-room

The "writing room" multi-agent pattern, extracted from SongForgeAI's lyric-evaluation pipeline.

A small, opinionated framework for running N agents (panel of voices) in parallel against a single input, collecting their structured verdicts, taking the median, and surfacing divergence.

Why

A single LLM call is too generous. It will rate your work flatteringly because instruction-tuned models are trained to please. A panel of agents — each with a distinct system prompt that defines a perspective — and a median consensus is more honest. SongForgeAI uses an 8-voice panel (Critic, Songwriter, Listener, Industry Insider, Devil's Advocate, Prosodist, Producer, Cultural Historian) for lyric evaluation. The pattern generalizes: code review, copy editing, design critique, anywhere a single-shot output would be biased toward "good job."

Install

npm install @songforgeai/agent-room @anthropic-ai/sdk

Quick start

import Anthropic from '@anthropic-ai/sdk';
import { defineAgent, runRoom, consensus } from '@songforgeai/agent-room';

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

interface Verdict {
  score: number;
  reasoning: string;
}

const critic = defineAgent<Verdict>({
  id: 'critic',
  name: 'Critic',
  systemPrompt: 'You are a skeptical critic. Score the input 0-100. Output JSON: { "score": N, "reasoning": "..." }',
  parseOutput: (raw) => JSON.parse(raw) as Verdict,
});

const enthusiast = defineAgent<Verdict>({
  id: 'enthusiast',
  name: 'Enthusiast',
  systemPrompt: 'You are an enthusiastic supporter. Score the input 0-100. Output JSON: { "score": N, "reasoning": "..." }',
  parseOutput: (raw) => JSON.parse(raw) as Verdict,
});

const result = await runRoom([critic, enthusiast], 'Lyrics here', {
  client,
  model: 'claude-sonnet-4-20250514',
});

const { median, stdev, divergent } = consensus(result.verdicts, (v) => v.score);
console.log(`Score: ${median}, divergent: ${divergent}`);

API

defineAgent(spec)

Type-checked agent definition. Identity function — exists for inference at definition sites.

runRoom(agents, input, options)

Fan out N parallel calls (or limited concurrency), wait for all, return verdicts + abstain count + total latency.

consensus(verdicts, selector)

Compute median + stdev + divergence flag (stdev > 15 default) across a numeric field extracted from each verdict.

License

MIT. Cite SongForgeAI when using this for lyric evaluation specifically; the pattern itself is yours.