Foundry QA Agent MCP Server

AI-powered QA testing via Model Context Protocol (MCP). Uses GLM-4.7 for test execution and Gemini 3 Flash for visual UX/UI critique.

Overview

This MCP server provides a single tool (run_qa_test) that executes browser-based QA tests using an autonomous AI agent. The agent:

• Navigates and interacts with your web application using Playwright
• Validates functionality and behavior
• Captures screenshots at key checkpoints
• Uses Gemini 3 Flash for visual UX/UI critique
• Reports findings with severity levels and reproduction steps

Setup

Option 1: Install from npm (Recommended)

npm install -g @diyor28/qa-mcp-server
npx playwright install chromium

Option 2: Install from Source

git clone https://github.com/iota-uz/foundry
cd foundry
pnpm install
pnpm build:qa-core && pnpm build:qa-mcp-server
cd packages/qa-core && npx playwright install chromium

Environment Variables

Required:

export CEREBRAS_API_KEY="your-cerebras-api-key"
export GEMINI_API_KEY="your-gemini-api-key"

Optional configuration:

export HEADLESS="true"                    # Run browser in headless mode
export ENABLE_TRACE="false"               # Capture Playwright traces
export TRACE_DIR="/tmp/qa-traces"         # Where to save traces
export MAX_ITERATIONS="15"                # Max agent loop iterations
export DEFAULT_VIEWPORT_WIDTH="1280"
export DEFAULT_VIEWPORT_HEIGHT="800"

Claude Desktop Configuration

Edit ~/Library/Application Support/Claude/claude_desktop_config.json:

If installed from npm:

{
  "mcpServers": {
    "foundry-qa": {
      "command": "npx",
      "args": ["-y", "@diyor28/qa-mcp-server"],
      "env": {
        "CEREBRAS_API_KEY": "your-key",
        "GEMINI_API_KEY": "your-key"
      }
    }
  }
}

If installed from source:

{
  "mcpServers": {
    "foundry-qa": {
      "command": "node",
      "args": ["/absolute/path/to/foundry/packages/qa-mcp-server/dist/index.js"],
      "env": {
        "CEREBRAS_API_KEY": "your-key",
        "GEMINI_API_KEY": "your-key"
      }
    }
  }
}

Restart Claude Desktop.

Usage

Basic Test

Ask Claude:

"Use run_qa_test to validate the login flow on http://localhost:3000"

Custom Scenario

Provide a detailed scenario:

"Run a QA test with this scenario:

{
  "baseUrl": "http://localhost:3000",
  "scenario": {
    "name": "Checkout Flow",
    "content": "1. Add item to cart\n2. Proceed to checkout\n3. Fill invalid card details\n4. Verify error message\n5. Fill valid card\n6. Complete order"
  }
}

UX/UI Critique

"Critique the UX/UI of http://localhost:3000/dashboard, focusing on layout clutter, color contrast, and navigation intuitiveness"

How It Works

Architecture

MCP Server (qa-mcp-server)
    ↓
QA Core Library (qa-core)
    ├── BrowserController (Playwright)
    ├── VisionAnalyzer (Gemini 3 Flash)
    ├── AgentLoop (GLM-4.7 via Cerebras)
    └── ReportBuilder

Test Execution Flow

1. Initialization: Launch Playwright browser with configured viewport
2. Agent Execution: GLM-4.7 agent follows scenario instructions, using tools:
   • navigate - Go to URLs
   • click - Click elements
   • fill - Fill form fields
   • screenshot - Capture screenshots
   • analyze_ui_ux - Request visual critique from Gemini
   • assert_visible, assert_text - Validate behavior
3. Visual Analysis: Gemini 3 Flash analyzes screenshots for UX/UI issues
4. Report Generation: Findings aggregated with severity levels
5. Artifact Collection: Screenshots and traces packaged in report

Context Management

The agent uses Foundry's @foundry/context library for intelligent context management:

• Token budgeting: Automatically fits within GLM-4.7's 200K context window
• Auto-compaction: Old tool outputs are pruned when context overflows
• Structured blocks: System prompt, scenario, browser state, history

Debugging

Enable Headed Mode

See the browser in action:

export HEADLESS="false"

Capture Traces

Enable Playwright tracing for detailed debugging:

export ENABLE_TRACE="true"
export TRACE_DIR="/tmp/qa-traces"

View traces with:

pnpm playwright show-trace /tmp/qa-traces/trace-*.zip

View Logs

Progress events are logged to stderr. When running locally:

node dist/index.js 2> qa-debug.log

Report Structure

The tool returns a JSON report with:

{
  "summary": "Test completed with 2 issue(s) found: 1 high, 1 medium.",
  "status": "issues_found",
  "duration": 45230,
  "findings": [
    {
      "severity": "high",
      "title": "Poor color contrast on login button",
      "description": "WCAG AA violation: contrast ratio 2.1:1 (requires 4.5:1)",
      "reproSteps": ["Navigate to /login", "Observe primary CTA button"],
      "category": "accessibility",
      "screenshotName": "login-page"
    }
  ],
  "artifacts": [
    {
      "type": "screenshot",
      "name": "login-page",
      "contentType": "image/png",
      "base64Data": "iVBORw0KGgoAAAANS..."
    }
  ],
  "metadata": {
    "scenarioName": "Login Flow",
    "baseUrl": "http://localhost:3000",
    "viewport": { "width": 1280, "height": 800 },
    "timestamp": "2026-01-27T12:00:00.000Z"
  }
}

Finding Severity Levels

• critical: Blocks core functionality, immediate fix required
• high: Significant issue affecting user experience
• medium: Noticeable issue, should be fixed
• low: Minor issue, cosmetic or edge case

Finding Categories

• functional: Core functionality bugs
• ux: User experience issues
• ui: Visual design issues
• accessibility: WCAG compliance violations

Integration with Backend

The qa-core library can be imported into the Foundry backend:

import { QaRunner } from '@diyor28/qa-core';

const runner = new QaRunner({
  browser: { headless: true, viewport: { width: 1280, height: 800 } },
  models: {
    cerebrasApiKey: process.env.CEREBRAS_API_KEY!,
    geminiApiKey: process.env.GEMINI_API_KEY!,
  },
  baseUrl: 'http://localhost:3000',
  scenario: {
    name: 'Test Scenario',
    content: '1. Navigate to homepage\n2. Verify title',
  },
});

const report = await runner.run();

This enables the existing QA service to use the same agent logic without MCP overhead.

Limitations

• Local browser only: Uses local Playwright, not Browserbase
• Single scenario per run: No test suite execution
• English only: Agent prompts in English

Troubleshooting

"Browser not found" error

Install Chromium:

pnpm playwright install chromium

"CEREBRAS_API_KEY is required" error

Set environment variables before starting:

export CEREBRAS_API_KEY="your-key"
export GEMINI_API_KEY="your-key"

Agent gets stuck

Try increasing max iterations:

export MAX_ITERATIONS="20"

Or provide more specific scenario instructions.

Screenshots not captured

The agent automatically captures screenshots. If none appear in artifacts, check:

1. Browser launched successfully
2. Navigation succeeded
3. Agent loop completed without errors

Development

Build and Watch

pnpm dev  # Run with tsx for development

Test Locally

# Start the server
node dist/index.js

# In another terminal, send a test request (requires MCP client)

License

Private - Foundry internal use only