Multiagent Consensus

A framework for running multi-agent consensus processes using multiple Large Language Models (LLMs). This library enables a "jury" of AI models to debate and reach consensus on queries, providing more robust and balanced responses.

Features

• 🤖 Multiple LLM Support: Compatible with various LLM providers through the Vercel AI SDK
• 🔄 Configurable Consensus Methods: Choose from majority, supermajority (75%), or unanimous agreement
• 🧠 Multi-round Debates: Models can debate in multiple rounds to refine their thinking
• 📊 Detailed Results: Get comprehensive metadata including confidence scores and processing time
• 🧪 Flexible Output: Customize output format (text, JSON) and content detail
• 🛠️ Highly Configurable: Set bias, system prompts, and customize debate parameters

Installation

npm install multiagent-consensus

Or with yarn:

yarn add multiagent-consensus

Basic Usage

import { ConsensusEngine } from 'multiagent-consensus';

// Create a consensus engine with your configuration
const engine = new ConsensusEngine({
  models: ['claude-3-haiku', 'gpt-4', 'palm-2'],
  consensusMethod: 'majority', // 'majority', 'supermajority', or 'unanimous'
  maxRounds: 2, // Maximum number of debate rounds
  output: {
    includeHistory: true, // Include debate history in result
    includeMetadata: true, // Include metadata in result
    format: 'text', // 'text' or 'json'
  },
});

// Run a consensus process
async function getConsensus() {
  const result = await engine.run('What is the best approach to solve climate change?');

  console.log('Final Answer:', result.answer);

  if (result.history) {
    console.log('Debate History:');
    result.history.forEach((round, i) => {
      console.log(`\nRound ${i + 1}:`);
      round.responses.forEach(response => {
        console.log(`${response.model}: ${response.response}`);
      });
    });
  }

  console.log('\nMetadata:', result.metadata);
}

getConsensus();

API Reference

ConsensusEngine

The main class for running consensus processes.

constructor(config: ConsensusConfig)

ConsensusConfig

| Parameter | Type | Description | Default | | --------------- | -------------------------------------------- | --------------------------------- | ---------- | | models | string[] | Array of model identifiers to use | Required | | consensusMethod | 'majority' | 'supermajority' | 'unanimous' | Method to determine consensus | 'majority' | | maxRounds | number | Maximum number of debate rounds | 3 | | output | OutputConfig | Output configuration | See below |

OutputConfig

| Parameter | Type | Description | Default | | --------------- | ---------------- | --------------------------- | ------- | | includeHistory | boolean | Include full debate history | false | | includeMetadata | boolean | Include metadata in result | true | | format | 'text' | 'json' | Output format | 'text' |

ConsensusResult

The result of a consensus process.

| Property | Type | Description | | -------- | -------------- | ------------------------------------------ | | answer | string | The final consensus answer | | models | string[] | Models that participated in the debate | | metadata | ResultMetadata | Information about the process | | history? | RoundData[] | Debate history (if includeHistory is true) |

ResultMetadata

| Property | Type | Description | | ---------------- | ---------------------- | ---------------------------------------------- | | totalTokens | number | Total tokens used across all models and rounds | | processingTimeMs | number | Total processing time in milliseconds | | rounds | number | Number of debate rounds conducted | | consensusMethod | string | Method used to determine consensus | | confidenceScores | Record<string, number> | Self-reported confidence per model |

Consensus Methods

Majority

Requires more than 50% of the models to agree on an answer. This is the most lenient consensus method and works well for straightforward queries.

Supermajority

Requires at least 75% of the models to agree. This provides a more stringent threshold for consensus, useful for complex or controversial topics.

Unanimous

Requires all models to agree completely. This is the strictest form of consensus and may require multiple debate rounds to achieve.

Setting Up Environment Variables

For using this package with LLM providers, you'll need to set up environment variables for API keys:

OPENAI_API_KEY=your_openai_key_here
ANTHROPIC_API_KEY=your_anthropic_key_here
COHERE_API_KEY=your_cohere_key_here

We recommend using dotenv for local development:

// In your application's entry point
import 'dotenv/config';

Examples

Custom Debate with Specific System Prompts

const engine = new ConsensusEngine({
  models: ['claude-3-sonnet', 'gpt-4', 'gpt-3.5-turbo'],
  consensusMethod: 'supermajority',
  maxRounds: 3,
  modelConfig: {
    'claude-3-sonnet': {
      systemPrompt: 'You are a scientific expert focused on evidence-based reasoning.',
      temperature: 0.5,
    },
    'gpt-4': {
      systemPrompt: 'You are a philosophical thinker who considers ethical implications.',
      temperature: 0.7,
    },
    'gpt-3.5-turbo': {
      systemPrompt: 'You are a practical problem-solver focusing on realistic solutions.',
      temperature: 0.3,
    },
  },
  output: {
    includeHistory: true,
    format: 'json',
  },
});

Using Bias Presets

const engine = new ConsensusEngine({
  models: ['claude-3-opus', 'gpt-4', 'llama-3'],
  consensusMethod: 'majority',
  biasPresets: ['centrist', 'progressive', 'conservative'],
  output: {
    includeHistory: true,
  },
});

Running the Examples

The package includes a JavaScript example to demonstrate functionality.

As a Package Consumer

When you've installed the published package as a dependency in your project:

# Install the package
npm install multiagent-consensus

# Copy the example file to your project
# Run the JavaScript example
node simple-consensus.js

As a Package Developer

When developing the package itself:

# From the package directory
npm run build         # Build the package first - this creates the dist directory
npm run example       # Run the JavaScript example

The build step is crucial as it compiles the TypeScript source files into JavaScript in the dist directory. The example imports code from this directory, so if you make changes to the source files, you'll need to rebuild the package before running the example again.

License

This project is licensed under the MIT License - see the LICENSE file for details.