@thinkhive/mcp-explainer

v3.0.1

Published

3 months ago

MCP server for ThinkHive Explainer API v3 - Run-centric architecture with facts vs inferences, deterministic linking, calibrated predictions, and AI agent observability

@thinkhive/mcp-explainer v2.0.1

MCP (Model Context Protocol) server for the ThinkHive Explainer API. Provides AI agents with access to trace analysis, RAG evaluation, hallucination detection, quality metrics, and business impact analytics.

What's New in v2.0.0

RAG Evaluation: Measure context relevance, groundedness, faithfulness, and citation accuracy
Hallucination Detection: Identify unsupported claims, factual errors, and contradictions
Quality Metrics: Track quality scores across agents with trend analysis
ROI Analytics: Calculate business impact, revenue at risk, and cost savings
Silent Failure Detection: Find cases where agents appear successful but failed to help
Fix Generation: AI-powered fix suggestions for failure patterns
Enhanced Prompts: Pre-built prompts for common analysis workflows

Features

25+ powerful tools for trace analysis, quality metrics, and business analytics
STDIO transport for Claude Code CLI integration
HTTP/SSE transport for web clients and external integrations
Comprehensive trace support: multi-turn conversations, tool calls, RAG context
Company isolation for multi-tenant security
Audit logging for compliance
Tier-based feature gates (free, starter, professional, enterprise)

Installation

npm install @thinkhive/mcp-explainer

Quick Start

Claude Code CLI

Add to your Claude Code configuration (~/.claude/claude_desktop_config.json):

{
  "mcpServers": {
    "thinkhive-explainer": {
      "command": "npx",
      "args": ["@thinkhive/mcp-explainer"],
      "env": {
        "THINKHIVE_API_KEY": "thk_your_api_key_here",
        "THINKHIVE_API_URL": "https://demo.thinkhive.ai"
      }
    }
  }
}

Programmatic Usage

import { startStdioServer } from '@thinkhive/mcp-explainer';

// Start STDIO server
await startStdioServer();

HTTP/SSE Transport

import express from 'express';
import { setupHttpTransport } from '@thinkhive/mcp-explainer';

const app = express();
app.use(express.json());

setupHttpTransport({
  app,
  basePath: '/mcp',
});

app.listen(3001, () => {
  console.log('MCP server running on http://localhost:3001');
});

Environment Variables

| Variable | Required | Default | Description | |----------|----------|---------|-------------| | THINKHIVE_API_KEY | Yes | - | Your ThinkHive API key | | THINKHIVE_API_URL | No | https://demo.thinkhive.ai | ThinkHive API URL |

Available Tools

Trace Analysis

| Tool | Description | Tier | |------|-------------|------| | trace_explain | Analyze with RAG eval & hallucination detection | Free | | trace_search | Search traces using natural language | Starter+ | | trace_get_explanation | Get stored explanation by trace ID | Free | | trace_patterns | Get failure patterns for an agent | Starter+ | | trace_stats | Get statistics for traces | Free | | trace_batch_analyze | Analyze multiple traces at once | Starter+ |

Quality Metrics (NEW)

| Tool | Description | Tier | |------|-------------|------| | quality_rag_evaluate | Evaluate RAG quality (groundedness, faithfulness) | Professional+ | | quality_hallucination_detect | Detect hallucinations in responses | Professional+ | | quality_get_scores | Get quality scores for trace/agent | Starter+ |

ROI Analytics (NEW)

| Tool | Description | Tier | |------|-------------|------| | roi_get_summary | Get ROI summary with revenue impact | Professional+ | | roi_get_correlations | Get performance-to-business correlations | Professional+ | | roi_calculate_impact | Calculate business impact for failures | Professional+ |

Failure Clusters

| Tool | Description | Tier | |------|-------------|------| | cluster_list | List failure clusters for an agent | Starter+ | | cluster_details | Get cluster details with examples | Starter+ |

Fix Generation (NEW)

| Tool | Description | Tier | |------|-------------|------| | fix_generate | Generate fix suggestions for failures | Professional+ | | fix_list | List generated fixes | Professional+ |

Silent Failures (NEW)

| Tool | Description | Tier | |------|-------------|------| | silent_failures_detect | Detect silent failures | Professional+ | | silent_failures_summary | Get silent failure summary | Professional+ |

Webhooks

| Tool | Description | Tier | |------|-------------|------| | webhook_list | List configured webhooks | Professional+ | | webhook_create | Create a new webhook | Professional+ | | webhook_delete | Delete a webhook | Professional+ | | webhook_test | Test a webhook | Professional+ |

Evaluation

| Tool | Description | Tier | |------|-------------|------| | eval_run | Run evaluation against golden dataset | Starter+ | | eval_list_sets | List evaluation sets | Starter+ | | eval_get_results | Get evaluation results | Starter+ |

Tool Examples

Analyze a Trace with Quality Metrics

{
  "trace": {
    "id": "trace-123",
    "userMessage": "What's the refund policy?",
    "agentResponse": "Our refund policy allows returns within 30 days...",
    "outcome": "success"
  },
  "options": {
    "evaluateRAG": true,
    "detectHallucinations": true,
    "computeBusinessImpact": true
  }
}

Evaluate RAG Quality

{
  "traceId": "trace-123",
  "question": "What's the refund policy?",
  "answer": "Our refund policy allows returns within 30 days...",
  "contexts": [
    "Customers may return items within 30 days of purchase...",
    "Refunds are processed within 5-7 business days..."
  ]
}

Get ROI Summary

{
  "agentId": "customer-support",
  "dateRange": {
    "from": "2024-01-01T00:00:00Z",
    "to": "2024-01-31T23:59:59Z"
  },
  "includeProjections": true
}

Detect Silent Failures

{
  "agentId": "sales-assistant",
  "signals": [
    "repeat_contact",
    "escalation",
    "negative_feedback"
  ]
}

Available Prompts

| Prompt | Description | |--------|-------------| | analyze-trace | Analyze a trace and provide insights | | debug-failure | Debug a failed interaction | | explain-cluster | Explain a failure cluster | | quality-report | Generate quality report for an agent | | roi-analysis | Generate ROI analysis | | hallucination-review | Review detected hallucinations |

Supported Models (December 2025)

The ThinkHive API backend uses the latest AI models for maximum accuracy:

Azure OpenAI

GPT-5 Series: gpt-5.2, gpt-5, gpt-5-pro, gpt-5-mini
GPT-4.1 Series: gpt-4.1, gpt-4.1-mini, gpt-4.1-nano
O-Series Reasoning: o4-mini, o3-pro, o3, o3-mini
Embeddings: text-embedding-3-large, text-embedding-3-small

Anthropic Claude (via Azure)

Claude 4.5 Series: claude-opus-4.5, claude-sonnet-4.5, claude-haiku-4.5

Google Gemini (Direct API)

Gemini 3: gemini-3-flash, gemini-3-flash-lite
Gemini 2.5: gemini-2.5-pro, gemini-2.5-flash

API Key Tiers

| Tier | Rate Limit | Features | |------|------------|----------| | Free | 10/min | Basic explainability | | Starter | 60/min | + Search, batch analysis, patterns, quality metrics | | Professional | 300/min | + RAG eval, hallucination detection, ROI analytics, webhooks | | Enterprise | 1,000/min | All features + dedicated support |

Security

API Key Authentication: Secure key validation with company scoping
Company Isolation: All queries scoped to authenticated company
Rate Limiting: Per-API-key rate limiting
Audit Logging: All tool calls logged for compliance

Troubleshooting

"API key required" error

Make sure THINKHIVE_API_KEY environment variable is set.

Rate limit exceeded

Wait for the rate limit window to reset (1 minute) or upgrade your tier.

Feature not available

Some features require higher tiers. Check the tier column in the tools table.

License

MIT