TruthGuard

Standardized grounding validation for tool-calling AI agents.

Detect when an agent's response contradicts the data returned by the tools it called — deterministically, without LLM-as-judge overhead.

The Problem

Most "hallucinations" in tool-calling agents are grounding failures — the agent calls a tool, gets accurate data, and then ignores it, miscalculates, or fabricates from empty results. The source of truth is already in the trace.

The Solution

TruthGuard extracts factual claims from the agent's response, cross-references them against tool outputs, and reports grounding failures with standardized codes — like OBD diagnostic codes for AI.

npm install truthguard-ai

Zero LLM calls. 30+ deterministic failure detectors. Runs in <50ms.

Quick Start — 3 Minutes

1. Evaluate a trace

import { TraceBuilder, GroundingEngine, generateReport } from 'truthguard-ai';

const trace = new TraceBuilder({ traceId: 'run-001' })
  .addUserInput('How many employees are on leave today?')
  .addToolCall('getLeaveRecords', { date: '2024-03-15' })
  .addToolOutput('getLeaveRecords', [
    { employeeId: 'E01', name: 'Ana Jovic', status: 'on_leave' },
    { employeeId: 'E02', name: 'Ivan Petrovic', status: 'on_leave' },
  ])
  .addFinalResponse('There are 3 employees on leave today.')  // ← Bug: says 3, data shows 2
  .build();

const engine = new GroundingEngine();
const report = engine.evaluate(trace);

console.log(report.groundingScore);       // 0.5
console.log(report.detectedFailures[0]);  // { type: 'grounding.data_ignored', severity: 'high' }

const { text } = generateReport(report);
console.log(text);

2. Add a CI quality gate

import { loadDataset, runDataset, evaluateGate, loadGateConfig } from 'truthguard-ai';

const entries = loadDataset('./test-cases.jsonl');
const result = runDataset(entries);
const gate = loadGateConfig('.ai-rcp-gate.yml');
const verdict = evaluateGate(result, gate);

if (!verdict.pass) {
  console.error(verdict.report);
  process.exit(1);
}

3. Monitor in production (proxy mode)

Works with any language — PHP, Python, Go, Java, Ruby, C#:

npx truthguard-ai observe --port 3001

Change your AI base URL and add your TruthGuard key using the composite key format (||):

# OpenAI — add TruthGuard key before your AI key, separated by ||
OPENAI_BASE_URL=http://localhost:3001/proxy/openai/v1
OPENAI_API_KEY=tg_live_your_key||sk-your-openai-key

# Anthropic
ANTHROPIC_BASE_URL=http://localhost:3001/proxy/anthropic
ANTHROPIC_API_KEY=tg_live_your_key||sk-ant-your-key

Your app works exactly the same — zero code changes. TruthGuard automatically splits the key, identifies your account, and forwards only the AI key to the provider.

Detection

30+ deterministic failure detectors across grounding, orchestration, and reasoning categories.

Examples:

• Fabrication from empty tool results
• Math errors against correct tool data
• Ignored or altered tool data in the response
• Entity mismatches and mix-ups

Features

Diagnostic Advisor

Every detected failure includes actionable diagnostics — root cause identification, evidence, and remediation guidance.

Policy Engine

Configure per-failure actions — block, warn, or observe:

import { wrapOpenAI, GroundingError } from 'truthguard-ai';
import OpenAI from 'openai';

const openai = wrapOpenAI(new OpenAI(), {
  mode: 'block',
  threshold: 0.85,
  policy: {
    rules: {
      'grounding.empty_fabrication': 'block',
      'grounding.math_error': 'warn',
      'reasoning.overconfident_language': 'observe',
    },
  },
});

Also available: wrapAnthropic for Anthropic SDK, wrapGemini for Google Gemini SDK.

import { wrapAnthropic } from 'truthguard-ai';
const client = wrapAnthropic(new Anthropic(), { mode: 'observe' });

import { wrapGemini } from 'truthguard-ai';
const model = wrapGemini(genAI.getGenerativeModel({ model: 'gemini-2.0-flash' }), { mode: 'observe' });

Baseline Regression Detection

import { createSnapshot, saveBaseline, loadBaseline, compareToBaseline } from 'truthguard-ai';

// Save after a known-good run
const snapshot = createSnapshot(result, 'v1.2-main');
saveBaseline('.ai-rcp-baseline.json', snapshot);

// Compare after changes
const comparison = compareToBaseline(newResult, snapshot);
if (!comparison.withinTolerance) {
  console.error('Regression detected:', comparison.report);
}

MCP Server (VS Code, Cursor, Windsurf, Zed, Claude Desktop)

Use TruthGuard directly from your IDE — no terminal needed.

Setup (one command):

npx truthguard-ai auth tg_live_your_key
npx truthguard-ai mcp-setup

Auto-detects installed IDEs and configures MCP. Restart your IDE after running.

1. In VS Code: Ctrl+Shift+P → "MCP: Open User Configuration"
2. Add this to mcp.json:

{
  "servers": {
    "truthguard": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "truthguard-ai", "mcp"]
    }
  }
}

3. Restart VS Code

Usage: In Copilot Chat, say: "Call truthguard verify_response with this trace: {...}"

8 tools available: verify_response, quick_check, check_trace_quality, list_rules, get_failure_info, evaluate_with_policy, get_live_traces, get_trace_report

Full setup guide: docs/getting-started.md

Express Middleware

import express from 'express';
import { groundingMiddleware, FileStore } from 'truthguard-ai';

const app = express();
app.post('/api/chat', groundingMiddleware({
  mode: 'warn',
  store: new FileStore('./traces/grounding.jsonl'),
  extractTrace: (req, res, body) => body.trace,
}));

CLI

npx truthguard-ai auth tg_live_your_key          # Save API key (once)
npx truthguard-ai mcp-setup                      # Auto-configure MCP for all IDEs
npx truthguard-ai debug trace.json               # Evaluate one trace
npx truthguard-ai run dataset.jsonl              # Batch dataset evaluation
npx truthguard-ai run dataset.jsonl --gate gate.yml  # CI quality gate
npx truthguard-ai observe --port 3001            # Start observe server + proxy

Auto-discovery: Set TRUTHGUARD_API_KEY env var and all SDK wrappers automatically send traces — no explicit client setup needed.

CI/CD Integration

GitHub Actions

# .github/workflows/truthguard-gate.yml
name: TruthGuard Quality Gate
on: [push, pull_request]

jobs:
  grounding-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
      - run: npm ci
      - run: npx truthguard-ai run test-cases.jsonl --gate .ai-rcp-gate.yml

Gate config (.ai-rcp-gate.yml)

name: "Grounding Quality Gate"
assertions:
  - metric: grounding_score
    operator: ">="
    threshold: 0.90
  - metric: failure_count
    operator: "<="
    threshold: 0
  - metric: pass_rate
    operator: ">="
    threshold: 1.0

How It Works

1. Extract factual claims from the agent's response
2. Match each claim against tool output data
3. Detect failure patterns across grounding, orchestration, and reasoning
4. Score overall grounding quality
5. Diagnose root causes with actionable remediation

No LLM calls. 100% deterministic. Configurable tolerances and multi-language support (13+ languages).

License

MIT