🎭 @maderatools/visual-ai

AI-assisted visual testing framework for Playwright with automatic bug reporting and Gemini integration.

🌟 Features

• 🛠️ MCP-Style Tools: Composable, chainable helpers for robust testing
• 📸 Smart Screenshots: Automatic capture with AI-ready context metadata
• 🔍 Real Visibility Checks: Verify elements are ACTUALLY visible, not just in DOM
• 🎯 Console & Network Monitoring: Capture errors, warnings, and API calls automatically
• 🤖 AI Integration: Built for Gemini Pro - generates tests on-demand
• 📋 Auto Bug Reports: Generates structured JSON for Claude CLI to fix bugs
• ⚡ Zero Config: Works out of the box with sensible defaults
• 🎨 Visual-First: Optimized for debugging complex UI interactions

📦 Installation

npm install --save-dev @maderatools/visual-ai @playwright/test playwright

Or with your preferred package manager:

yarn add -D @maderatools/visual-ai @playwright/test playwright
pnpm add -D @maderatools/visual-ai @playwright/test playwright

🚀 Quick Start

1. Create a Test

// tests/example.spec.js
import { test, expect } from '@playwright/test';
import {
  VisualHelpers,
  ScreenshotManager,
  ConsoleCapture,
  NetworkMonitor
} from '@maderatools/visual-ai';

test.describe('My Feature', () => {
  let consoleCapture, networkMonitor;

  test.beforeEach(async ({ page }) => {
    // Setup monitoring
    consoleCapture = new ConsoleCapture();
    await consoleCapture.setup(page);

    networkMonitor = new NetworkMonitor();
    await networkMonitor.startMonitoring(page);

    await page.goto('https://your-app.com');
  });

  test('Button opens modal', async ({ page }) => {
    // Wait for page fully loaded
    await VisualHelpers.waitForFullLoad(page);

    // Capture initial state
    await ScreenshotManager.captureStep(page, 'page_loaded');

    // Click button with real visibility check
    await VisualHelpers.clickVisible(page, '.open-modal-btn');

    // Verify modal is REALLY visible
    const modalVisible = await VisualHelpers.verifyModalVisible(page, '.modal');
    expect(modalVisible).toBeTruthy();

    // Capture final state
    await ScreenshotManager.captureStep(page, 'modal_opened');
  });
});

2. Run Tests

npx playwright test

3. Get AI-Ready Bug Reports

If tests fail, a JSON report is automatically generated in tests/reports/ with:

• Screenshots with context
• Console errors & warnings
• Network failures
• Affected files with line numbers
• Root cause hypothesis
• Suggested fixes

Copy the JSON → Paste to Claude CLI → Bugs get fixed! 🎯

🛠️ API Reference

VisualHelpers

Interaction helpers with REAL visibility verification.

import { VisualHelpers } from '@maderatools/visual-ai';

// Wait for page fully loaded with custom checks
await VisualHelpers.waitForFullLoad(page, [
  () => typeof window.myApp !== 'undefined',
  () => document.querySelectorAll('.item').length > 0
]);

// Click with visibility verification
await VisualHelpers.clickVisible(page, '.button');

// Scroll element into view
await VisualHelpers.scrollToVisible(page, '.element');

// Verify modal is really visible
const visible = await VisualHelpers.verifyModalVisible(page, '.modal');

// Wait for animations to complete
await VisualHelpers.waitForAnimations(page);

// Check if element is interactable (not disabled/covered)
const canClick = await VisualHelpers.isInteractable(page, '.button');

// Fill input with verification
await VisualHelpers.fillAndVerify(page, '#email', '[email protected]');

// Wait for element to disappear
await VisualHelpers.waitForDisappear(page, '.loading');

// Wait for text to appear
await VisualHelpers.waitForText(page, '.message', 'Success!');

ScreenshotManager

Capture screenshots with AI-ready context.

import { ScreenshotManager } from '@maderatools/visual-ai';

// Capture a step
await ScreenshotManager.captureStep(page, 'step_name', {
  note: 'Description for AI analysis'
});

// Capture before/after comparison
await ScreenshotManager.captureComparison(
  page,
  'button_click',
  null,
  async () => await page.click('.button')
);

// Capture specific element
await ScreenshotManager.captureElement(page, '.modal', 'modal_content');

// Capture full page
await ScreenshotManager.captureFullPage(page, 'full_page');

// Get all screenshots for a test
const screenshots = ScreenshotManager.getTestScreenshots('test_id');

ConsoleCapture

Monitor browser console automatically.

import { ConsoleCapture } from '@maderatools/visual-ai';

const consoleCapture = new ConsoleCapture();
await consoleCapture.setup(page);

// Get errors
const errors = consoleCapture.getErrors();

// Get structured report
const report = consoleCapture.getStructuredReport();

// Detect error patterns
const patterns = consoleCapture.detectErrorPatterns();

// Print summary
consoleCapture.printSummary();

NetworkMonitor

Track API calls and network activity.

import { NetworkMonitor } from '@maderatools/visual-ai';

const networkMonitor = new NetworkMonitor();
await networkMonitor.startMonitoring(page, {
  apiPatterns: ['/api/', '/v1/']
});

// Get API calls
const apiCalls = networkMonitor.getApiCalls();

// Get failed requests
const failures = networkMonitor.getFailedRequests();

// Export HAR file
await networkMonitor.exportHAR('network-activity.har');

ReportBuilder

Generate JSON reports for AI bug fixing.

import { ReportBuilder } from '@maderatools/visual-ai';

const reportBuilder = new ReportBuilder();

const report = await reportBuilder.buildReport(
  testResult,
  {
    screenshots,
    consoleLogs: consoleCapture.getStructuredReport(),
    networkLogs: networkMonitor.getStructuredReport()
  }
);

await reportBuilder.exportForClaude(report, 'bug-report.json');

🤖 AI Integration (Gemini)

This framework is designed to work with Gemini Pro for on-demand test generation.

Workflow:

1. User: "Test the login modal"
2. Gemini: Reads examples, generates custom test using the tools
3. Gemini: Runs test via Playwright
4. If fail: Gemini analyzes screenshots, generates JSON for Claude
5. User: Copy JSON → Paste to Claude CLI
6. Claude: Reads context, fixes bugs, commits changes

See docs/GEMINI-GUIDE.md for complete instructions for AI.

📚 Examples

Check examples/ for complete test templates:

• extension-test.spec.js - Chrome extension testing
• dragdrop-test.spec.js - Drag-and-drop interactions
• api-test.spec.js - Backend API workflows

⚙️ Configuration

Using Default Config

// playwright.config.js
import { defineConfig } from '@playwright/test';
import { defaultPlaywrightConfig } from '@maderatools/visual-ai';

export default defineConfig({
  ...defaultPlaywrightConfig,
  // Override what you need
  testDir: './my-tests',
  use: {
    ...defaultPlaywrightConfig.use,
    baseURL: 'https://my-app.com'
  }
});

Custom Config

// playwright.config.js
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  timeout: 60000,
  use: {
    viewport: { width: 1920, height: 1080 },
    slowMo: 500, // Slow down for visual debugging
    screenshot: 'on',
    video: 'retain-on-failure',
    trace: 'retain-on-failure'
  }
});

🎯 Best Practices

1. Always use VisualHelpers instead of plain Playwright methods
2. Capture screenshots at every important step
3. Setup monitoring in beforeEach (ConsoleCapture + NetworkMonitor)
4. Verify REAL visibility not just DOM presence
5. Wait for animations before assertions
6. Use custom checks in waitForFullLoad() for your app's specific state

🤝 Contributing

Contributions welcome! Please read CONTRIBUTING.md first.

Development Setup

git clone https://github.com/mussonking/madera_tools.git
cd @maderatools/visual-ai
npm install

📄 License

MIT © Maderatools

🙏 Acknowledgments

• Built on top of Playwright
• Inspired by visual regression testing best practices
• Designed for AI-assisted development workflows

📞 Support

• 🐛 Report a bug
• 💡 Request a feature
• 📖 Read the docs

Made with ❤️ for the Playwright community