LaaJ Judge MCP Server

TypeScript MCP (Model Context Protocol) server for the Universal AI Observer Layer. Exposes the 5-tier KPI evaluation engine as tools for AI assistants like Claude Desktop, VS Code Copilot, Cursor, and any MCP-compatible client.

Architecture

AI Assistant (Claude Desktop / VS Code / Cursor)
        |
        | MCP Protocol (stdio / Streamable HTTP)
        v
  MCP Server (TypeScript, this package)
        |
        | HTTP/JSON
        v
  Rule Engine (FastAPI, port 8001)
        |
        +--- PII Detector (Presidio)
        +--- Secret Detector (Gitleaks rules)
        +--- Code Scanner (Semgrep)
        +--- EU AI Act Detector (Reg. 2024/1689)
        +--- Toxicity Detector (Detoxify)
        +--- G-Eval Scorer (LLM-as-a-Judge)
        +--- QAG Factuality Scorer
        +--- OPA Policy Engine
        +--- Trust Score Calculator

Tools

| Tool | Description | |------|-------------| | analyze_text | Full 5-tier KPI analysis (Faithfulness, Bias, Compliance, Robustness, Latency) | | check_compliance | EU AI Act (Regulation 2024/1689) compliance check | | compare_responses | Side-by-side comparison of two AI responses | | health_check | Rule engine connectivity check |

Resources

| URI | Description | |-----|-------------| | laaj://kpi-tiers | KPI tier definitions, weights, detectors, and scoring formula | | laaj://use-cases | Use-case profiles (general, code, legal, medical) |

Prompts

| Name | Description | |------|-------------| | evaluate-ai-output | Guided evaluation using the 5-tier framework | | check-eu-compliance | Guided EU AI Act compliance check |

Installation

npm install -g @wamocon/laaj-judge-mcp

Or use directly with npx:

npx @wamocon/laaj-judge-mcp

Usage with Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "laaj-judge": {
      "command": "npx",
      "args": ["@wamocon/laaj-judge-mcp"],
      "env": {
        "RULE_ENGINE_URL": "http://localhost:8001"
      }
    }
  }
}

Usage with VS Code

Add to .vscode/mcp.json:

{
  "servers": {
    "laaj-judge": {
      "command": "npx",
      "args": ["@wamocon/laaj-judge-mcp"],
      "env": {
        "RULE_ENGINE_URL": "http://localhost:8001"
      }
    }
  }
}

Usage with Docker

docker build -t laaj-judge-mcp ./mcp-server
docker run --rm -e RULE_ENGINE_URL=http://host.docker.internal:8001 laaj-judge-mcp

CLI Options

laaj-judge-mcp [options]

Options:
  --transport <stdio|http>    Transport mode (default: stdio)
  --port <number>             HTTP port for http transport (default: 3100)
  --rule-engine-url <url>     Rule engine URL (default: http://localhost:8001)
  --timeout <ms>              Request timeout (default: 30000)
  --version                   Show version
  --help                      Show help

Environment Variables

| Variable | Default | Description | |----------|---------|-------------| | RULE_ENGINE_URL | http://localhost:8001 | URL of the UAOL rule engine | | MCP_TRANSPORT | stdio | Transport mode | | MCP_HTTP_PORT | 3100 | HTTP port | | MCP_REQUEST_TIMEOUT_MS | 30000 | Request timeout in ms |

Development

cd mcp-server
npm install
npm run dev          # Watch mode
npm run build        # Production build
npm run typecheck    # Type checking
npm run lint         # ESLint

KPI Tiers

| Tier | Name | Weight | Description | |------|------|--------|-------------| | A | Faithfulness | 35% | Factual accuracy and source grounding | | B | Bias | 15% | Demographic, cultural, ideological bias detection | | C | Compliance | 25% | EU AI Act (2024/1689) adherence | | D | Robustness | 15% | Adversarial resilience, code security | | E | Latency | 10% | Response time characteristics |

Publishing to npm

npm login
npm publish --access public

License

MIT