@majkapp/bot-plugin-server

Playwright-based automation and testing framework for MAJK plugins. Test your plugin UIs using semantic data-majk-* attributes for stable, maintainable tests.

✨ What's New

• 🎯 Real Plugin Functions - Automatically loads and invokes your actual plugin functions (no mocks needed!)
• 📋 Console Log Capture - Captures all client-side console.log, console.error, etc. with timestamps and source locations
• 🐛 Error Tracking - Auto-captures page errors with stack traces and auto-screenshots on errors
• 🎮 Interactive Mode - Open your plugin in a browser for manual testing while streaming logs in real-time
• 📁 Organized Output - Logs saved alongside screenshots in per-test directories

Quick Start with npx

The fastest way to test your MAJK plugin is with npx:

Example 1: Basic Test with Auto-Managed Server

npx @majkapp/bot-plugin-server \
  --script ./my-test.js \
  --plugin ./my-plugin \
  --headed

What this does:

1. 🚀 Automatically starts a plugin server for your plugin
2. 🌐 Launches Chrome browser (visible because of --headed)
3. 📝 Runs your test script
4. 📸 Saves screenshots to ./screenshots/
5. 🛑 Cleans up (stops server and browser)

Where things go:

• Screenshots: ./screenshots/ (configurable with --screenshots)
• Logs: ./logs/ (configurable with --logs)
• Test results: Printed to console

Example 2: Complete Test with Mock Handlers

First, create your test fixtures:

fixtures/handlers.js - Mock function handlers:

module.exports = {
  async getUserProfile({ userId }) {
    return {
      success: true,
      data: {
        id: userId,
        name: 'Test User',
        email: '[email protected]',
        role: 'admin'
      }
    };
  },

  async saveUserProfile({ userId, profile }) {
    console.log('Saving profile:', profile);
    return {
      success: true,
      data: { saved: true, timestamp: Date.now() }
    };
  },

  async deleteUser({ userId }) {
    return {
      success: true,
      data: { deleted: true }
    };
  }
};

tests/user-profile-test.js - Your test script:

module.exports = async function(bot) {
  console.log('🧪 Testing user profile page...');

  // Navigate to plugin
  await bot.navigate();
  await bot.screenshot('01-home');

  // Navigate to profile page (if using hash routing)
  const page = bot.getPage();
  await page.goto(`${page.url()}#/profile`);
  await bot.waitForState('loaded', { timeout: 5000 });
  await bot.screenshot('02-profile-loaded');

  // Edit profile
  await bot.click('editButton');
  await bot.type('nameField', 'Updated Name');
  await bot.type('emailField', '[email protected]');
  await bot.screenshot('03-profile-edited');

  // Save changes
  await bot.click('saveButton');
  await bot.waitForState('success', { timeout: 3000 });
  await bot.screenshot('04-profile-saved');

  console.log('✅ Profile test completed!');
};

Run the complete test:

npx @majkapp/bot-plugin-server \
  --script ./tests/user-profile-test.js \
  --plugin ./my-plugin \
  --handlers ./fixtures/handlers.js \
  --screenshots ./test-results/screenshots \
  --slow-mo 300 \
  --headed

What this does:

1. 🚀 Starts plugin server with your plugin
2. 🔧 Loads mock handlers from ./fixtures/handlers.js
3. 🌐 Opens Chrome browser (visible, slowed down 300ms per action)
4. 📝 Runs your test: navigate → edit → save → verify
5. 📸 Saves 4 screenshots to ./test-results/screenshots/:
   • 01-home.png - Initial state
   • 02-profile-loaded.png - Profile page loaded
   • 03-profile-edited.png - After editing fields
   • 04-profile-saved.png - Success state
6. ✅ Prints success/failure to console
7. 🛑 Cleans up everything

Example 3: Form Validation Test

tests/form-validation.js:

module.exports = async function(bot) {
  console.log('🧪 Testing form validation...');

  await bot.navigate();
  const page = bot.getPage();
  await page.goto(`${page.url()}#/contact`);
  await bot.screenshot('01-empty-form');

  // Test 1: Submit empty form (should show errors)
  console.log('  → Testing empty form submission...');
  await bot.click('submitButton');
  await page.waitForTimeout(500);
  await bot.screenshot('02-validation-errors');

  // Verify errors appeared
  const nameError = await page.locator('[data-majk-error="name"]').isVisible();
  const emailError = await page.locator('[data-majk-error="email"]').isVisible();
  console.log(`  → Name error visible: ${nameError}`);
  console.log(`  → Email error visible: ${emailError}`);

  // Test 2: Fill form with valid data
  console.log('  → Filling form with valid data...');
  await bot.type('name', 'John Doe');
  await bot.type('email', '[email protected]');
  await bot.type('message', 'This is a test message!');
  await bot.screenshot('03-form-filled');

  // Test 3: Submit valid form
  console.log('  → Submitting valid form...');
  await bot.click('submitButton');
  await bot.waitForState('success', { timeout: 5000 });
  await bot.screenshot('04-success');

  console.log('✅ Validation test completed!');
};

Run it:

npx @majkapp/bot-plugin-server \
  --script ./tests/form-validation.js \
  --plugin ./my-plugin \
  --headless

Output:

🚀 Starting plugin server for _/my-plugin
🚀 MAJK Plugin Server running on http://localhost:54123
   Serving 1 plugin(s)
✅ Server started on http://localhost:54123
🌐 Starting browser...
✅ Browser started
📍 Navigating to http://localhost:54123/plugins/_/my-plugin/ui
✅ Page loaded
📸 Taking screenshot: screenshots/01-empty-form.png
✅ Screenshot saved
🧪 Testing form validation...
  → Testing empty form submission...
🖱️  Clicking element: submitButton
✅ Clicked: submitButton
📸 Taking screenshot: screenshots/02-validation-errors.png
✅ Screenshot saved
  → Name error visible: true
  → Email error visible: true
  → Filling form with valid data...
⌨️  Typing into field: name
✅ Typed into: name
⌨️  Typing into field: email
✅ Typed into: email
⌨️  Typing into field: message
✅ Typed into: message
📸 Taking screenshot: screenshots/03-form-filled.png
✅ Screenshot saved
  → Submitting valid form...
🖱️  Clicking element: submitButton
✅ Clicked: submitButton
⏳ Waiting for state: success
✅ State reached: success
📸 Taking screenshot: screenshots/04-success.png
✅ Screenshot saved
✅ Validation test completed!
🛑 Stopping...
✅ Browser closed
✅ Server stopped

✅ Test completed successfully!

Example 4: Multi-Step Wizard Test

tests/wizard-flow.js:

module.exports = async function(bot) {
  console.log('🧪 Testing wizard flow...');

  await bot.navigate();
  const page = bot.getPage();
  await page.goto(`${page.url()}#/wizard`);
  await bot.waitForState('step-1');
  await bot.screenshot('01-step-1');

  // Step 1
  console.log('  → Completing step 1...');
  await bot.type('firstName', 'John');
  await bot.type('lastName', 'Doe');
  await bot.click('nextButton');
  await bot.waitForState('step-2');
  await bot.screenshot('02-step-2');

  // Step 2
  console.log('  → Completing step 2...');
  await bot.type('email', '[email protected]');
  await bot.type('phone', '555-1234');
  await bot.click('nextButton');
  await bot.waitForState('step-3');
  await bot.screenshot('03-step-3');

  // Step 3
  console.log('  → Completing step 3...');
  await bot.click('agreeCheckbox');
  await bot.click('completeButton');
  await bot.waitForState('completed', { timeout: 5000 });
  await bot.screenshot('04-completed');

  console.log('✅ Wizard test completed!');
};

Run it:

npx @majkapp/bot-plugin-server \
  --script ./tests/wizard-flow.js \
  --plugin ./my-plugin \
  --handlers ./fixtures/handlers.js \
  --middleware ./fixtures/middleware.js \
  --screenshots ./test-results/wizard \
  --slow-mo 500 \
  --headed \
  --verbose

Example 5: CI/CD Integration

.github/workflows/test.yml:

name: Plugin Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-node@v3
        with:
          node-version: '18'

      - name: Install dependencies
        run: |
          cd my-plugin
          npm install
          npm run build

      - name: Install Playwright
        run: npx playwright install chromium

      - name: Run bot tests
        run: |
          npx @majkapp/bot-plugin-server \
            --script ./tests/all-tests.js \
            --plugin ./my-plugin \
            --handlers ./fixtures/handlers.js \
            --headless \
            --screenshots ./test-results/screenshots

      - name: Upload screenshots
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: test-screenshots
          path: test-results/screenshots/

CLI Options Reference

Required Options

| Option | Description | Example | |--------|-------------|---------| | --script <path> | Path to test script file | --script ./tests/my-test.js |

Plugin Source (choose one)

| Option | Description | Example | |--------|-------------|---------| | --plugin <path> | Auto-start server for this plugin | --plugin ./my-plugin | | --url <url> | Connect to existing server | --url http://localhost:3000 |

Optional Configuration

| Option | Description | Default | |--------|-------------|---------| | --handlers <path> | Function handlers file | None | | --middleware <path> | Middleware file | None | | --screenshots <path> | Screenshots directory | ./screenshots | | --logs <path> | Logs directory | ./logs | | --asset-path-prefix <path> | Asset path prefix | /plugin-screens | | --headless | Run browser in headless mode | true | | --headed | Show browser window | N/A | | --slow-mo <ms> | Slow down actions (milliseconds) | 0 | | --verbose | Show detailed output | false | | -h, --help | Show help message | N/A |

Writing Test Scripts

Your test script should export a function that receives a bot instance:

// Simple test
module.exports = async function(bot) {
  await bot.navigate();
  await bot.click('myButton');
  await bot.screenshot('after-click');
};

Or using ES modules:

export default async function(bot) {
  await bot.navigate();
  await bot.click('myButton');
  await bot.screenshot('after-click');
};

Bot API

| Method | Description | Example | |--------|-------------|---------| | navigate(url?) | Navigate to plugin (or custom URL) | await bot.navigate() | | click(id) | Click element by data-majk-id | await bot.click('submitButton') | | type(field, value) | Type into data-majk-field | await bot.type('email', '[email protected]') | | waitForState(state, opts?) | Wait for data-majk-state | await bot.waitForState('success', { timeout: 5000 }) | | screenshot(name) | Capture screenshot | await bot.screenshot('my-screenshot') | | getPage() | Get Playwright Page for advanced operations | const page = bot.getPage() |

Advanced Usage

For more control, access the Playwright page directly:

module.exports = async function(bot) {
  const page = bot.getPage();

  // Custom navigation
  await page.goto('https://example.com');

  // Custom selectors
  await page.click('.my-custom-class');

  // Custom waits
  await page.waitForSelector('.loaded', { timeout: 10000 });

  // Extract data
  const text = await page.textContent('[data-majk-id="message"]');
  console.log('Message:', text);

  // Network interception
  await page.route('**/api/**', route => {
    route.fulfill({
      status: 200,
      body: JSON.stringify({ mocked: true })
    });
  });

  // Standard bot API still works
  await bot.click('submitButton');
  await bot.screenshot('result');
};

UI Development Guide

To make your plugin testable, add data-majk-* attributes to your components:

<Button
  data-majk-action="submit"
  data-majk-id="submitButton"
  onPress={handleSubmit}
>
  Submit
</Button>

<Input
  label="Email"
  data-majk-field="email"
  data-majk-id="emailInput"
  value={email}
  onChange={setEmail}
/>

{submitted && (
  <div data-majk-state="success" data-majk-id="successMessage">
    Success!
  </div>
)}

{error && (
  <div data-majk-error="validation">
    {error}
  </div>
)}

See DEVELOPER_GUIDE.md for complete UI development guide.

Installation (for project usage)

npm install --save-dev @majkapp/bot-plugin-server

Then use programmatically:

const { runBot } = require('@majkapp/bot-plugin-server');

await runBot({
  plugin: {
    dir: './my-plugin',
    assetPathPrefix: '/plugin-screens'
  },
  output: {
    screenshots: './screenshots'
  },
  browser: {
    headless: true
  }
}, async (bot) => {
  await bot.navigate();
  await bot.click('submitButton');
  await bot.screenshot('result');
});

Common Patterns

Pattern 1: Test Suite with Multiple Tests

tests/all-tests.js:

const testLogin = require('./login-test');
const testProfile = require('./profile-test');
const testSettings = require('./settings-test');

module.exports = async function(bot) {
  console.log('🧪 Running test suite...\n');

  try {
    console.log('1️⃣  Running login test...');
    await testLogin(bot);
    console.log('✅ Login test passed\n');

    console.log('2️⃣  Running profile test...');
    await testProfile(bot);
    console.log('✅ Profile test passed\n');

    console.log('3️⃣  Running settings test...');
    await testSettings(bot);
    console.log('✅ Settings test passed\n');

    console.log('🎉 All tests passed!');
  } catch (error) {
    console.error('❌ Test failed:', error.message);
    throw error;
  }
};

Pattern 2: Reusable Test Helpers

tests/helpers.js:

async function login(bot, username, password) {
  const page = bot.getPage();
  await page.goto(`${page.url()}#/login`);
  await bot.type('username', username);
  await bot.type('password', password);
  await bot.click('loginButton');
  await bot.waitForState('logged-in', { timeout: 5000 });
}

async function logout(bot) {
  await bot.click('userMenu');
  await bot.click('logoutButton');
  await bot.waitForState('logged-out', { timeout: 5000 });
}

module.exports = { login, logout };

Usage:

const { login, logout } = require('./helpers');

module.exports = async function(bot) {
  await bot.navigate();

  await login(bot, 'testuser', 'password');
  await bot.screenshot('logged-in');

  // Do something as logged-in user
  await bot.click('profileButton');

  await logout(bot);
  await bot.screenshot('logged-out');
};

Debugging Tips

1. Use --headed and --slow-mo

npx @majkapp/bot-plugin-server \
  --script ./tests/my-test.js \
  --plugin ./my-plugin \
  --headed \
  --slow-mo 1000

This lets you watch the browser in slow motion!

2. Add Verbose Logging

npx @majkapp/bot-plugin-server \
  --script ./tests/my-test.js \
  --plugin ./my-plugin \
  --verbose

3. Capture Screenshots at Every Step

module.exports = async function(bot) {
  await bot.navigate();
  await bot.screenshot('step-1-navigate');

  await bot.type('email', '[email protected]');
  await bot.screenshot('step-2-email-filled');

  await bot.click('submitButton');
  await bot.screenshot('step-3-submitted');

  await bot.waitForState('success');
  await bot.screenshot('step-4-success');
};

4. Use Browser Console

module.exports = async function(bot) {
  const page = bot.getPage();

  // Listen to console
  page.on('console', msg => console.log('🖥️  Browser:', msg.text()));
  page.on('pageerror', err => console.error('❌ Page error:', err.message));

  // Rest of your test
  await bot.navigate();
  await bot.click('submitButton');
};

Examples

See the examples/ directory for complete working examples:

• test-button-clicks.js - Button interactions and counters
• test-form-filling.js - Form validation and submission
• test-plugin-home.js - Basic navigation
• wizard-screenshot.js - Screenshot capture

Dependencies

• @majkapp/majk-plugin-server ^1.1.0 - Plugin server
• playwright ^1.40.0 - Browser automation
• @playwright/test ^1.40.0 - Testing utilities

License

MIT

Automated Testing Framework

For automated plugin testing with test runners, use @majkapp/plugin-test which provides a higher-level testing API built on top of this bot framework.

Quick Example with @majkapp/plugin-test

Install:

npm install --save-dev @majkapp/plugin-test

Write UI tests with uiTest:

const { uiTest } = require('@majkapp/plugin-test');

// Configure once
uiTest.configure({
  plugin: { dir: process.cwd() },
  browser: { headless: true, slowMo: 200 }
});

// Write tests
uiTest('switch to Wizards tab', async (bot) => {
  await bot.navigate();
  await bot.screenshot('01-home-buttons-tab');

  const page = bot.getPage();
  await page.click('button[data-key="wizards"]');
  await page.waitForTimeout(500);
  await bot.screenshot('02-wizards-tab-active');

  const wizardsContent = await page.locator('text=Wizard Examples').isVisible();
  console.log(`✅ Wizards tab content visible: ${wizardsContent}`);
});

uiTest('navigate to Conversations Sample', async (bot) => {
  await bot.navigate();
  await bot.screenshot('01-home-page');

  const page = bot.getPage();
  await page.click('a:has-text("Conversations Sample")');
  await page.waitForTimeout(1000);
  await bot.screenshot('02-conversations-sample-loaded');

  const url = page.url();
  console.log(`✅ Current URL: ${url}`);
});

UI Testing with Mock Data

The bot framework supports mock handlers for plugin functions:

const { runBot } = require('@majkapp/bot-plugin-server');

await runBot({
  plugin: { dir: './my-plugin' },
  handlers: {
    async getConversations({ limit = 100, offset = 0 }) {
      return [
        {
          id: 'conv-1',
          title: 'Plugin Architecture Discussion',
          createdAt: new Date('2025-11-10').toISOString(),
          updatedAt: new Date('2025-11-12').toISOString(),
          messageCount: 15
        },
        {
          id: 'conv-2',
          title: 'Testing Framework Design',
          createdAt: new Date('2025-11-11').toISOString(),
          updatedAt: new Date('2025-11-13').toISOString(),
          messageCount: 8
        }
      ];
    },

    async getConversationMessages({ conversationId }) {
      return [
        {
          id: 'msg-1',
          role: 'user',
          content: 'Can you help me understand how plugins work?',
          timestamp: new Date().toISOString()
        },
        {
          id: 'msg-2',
          role: 'assistant',
          content: 'Of course! MAJK plugins are modular extensions.',
          timestamp: new Date().toISOString()
        }
      ];
    }
  },
  output: { screenshots: './screenshots' },
  browser: { headless: true }
}, async (bot) => {
  await bot.navigate();

  const page = bot.getPage();
  await page.click('a:has-text("Conversations Sample")');
  await page.waitForTimeout(1000);
  await bot.screenshot('conversations-loaded');

  // Verify mock data appeared
  const conversationItems = await page.locator('[data-majk-id="conversationItem"]').count();
  console.log(`✅ Found ${conversationItems} conversation items`);

  // Click first conversation
  await page.click('[data-majk-id="conversationItem"]:first-child');
  await page.waitForTimeout(500);
  await bot.screenshot('conversation-selected');
});

How Mock Handlers Work:

1. Configure handlers in BotConfig.handlers
2. When UI makes API calls like /plugins/{org}/{id}/api/getConversations
3. The bot server's functionInvoker intercepts and calls your handler
4. Mock data is returned to the UI
5. You can verify the UI displays the mock data correctly

Screenshot Management

uiTest automatically manages screenshots with organized naming:

uiTest('test with multiple screenshots', async (bot) => {
  await bot.navigate();
  // Creates: screenshots/test-with-multiple-screenshots/00-home.png
  await bot.screenshot('home');

  await bot.click('submitButton');
  // Creates: screenshots/test-with-multiple-screenshots/01-after-submit.png
  await bot.screenshot('after-submit');

  // Error screenshots are automatic:
  try {
    await bot.click('nonExistentButton');
  } catch (error) {
    // Creates: screenshots/test-with-multiple-screenshots/02-error-click-nonExistentButton.png
  }
});

Running Tests

With @majkapp/plugin-test CLI:

# Run all tests
npx majk-test

# Run only UI tests
npx majk-test --type ui

# Run specific test file
npx majk-test tests/plugin/ui/e2e/my-test.test.js

# Generate HTML report
npx majk-test --reporter html --output test-results.html

Test output includes:

• ✅ Pass/fail status for each test
• 📸 Screenshot paths
• ⏱️ Test duration
• 📊 Summary statistics

Complete Testing Workflow

1. Setup (tests/plugin/ui/e2e/my-flow.test.js):

const { uiTest } = require('@majkapp/plugin-test');

uiTest.configure({
  plugin: { dir: process.cwd() },
  browser: { headless: true, slowMo: 200 }
});

2. Write tests with mock data:

const mockHandlers = {
  async myFunction(params) {
    return { success: true, data: { result: 'mocked' } };
  }
};

uiTest('test with mocks', async (bot) => {
  // Configure handlers
  bot.configure({ handlers: mockHandlers });

  await bot.navigate();
  await bot.click('triggerButton');
  await bot.screenshot('after-trigger');

  // Verify UI shows mock data
  const page = bot.getPage();
  const text = await page.textContent('[data-majk-id="result"]');
  console.log(`✅ Result: ${text}`);
});

3. Run tests:

cd my-plugin
npx majk-test --type ui --env e2e

4. Review results:

• Console output shows pass/fail
• Screenshots in screenshots/{test-name}/ directory
• HTML report (if using --reporter html)

Plugin Inspection

The bot framework includes plugin inspection tools for debugging:

# Inspect plugin structure
npx bot-plugin-server inspect ./my-plugin

# Test function execution
npx bot-plugin-server inspect ./my-plugin --function myFunction --params '{"key":"value"}'

# With authentication
npx bot-plugin-server inspect ./my-plugin \
  --function myFunction \
  --auth-endpoint http://localhost:3000/auth

Inspection shows:

• 📦 Plugin metadata (name, version, org, ID)
• 🔧 Available functions with input/output schemas
• 🎨 UI screens and routes
• 🔌 API routes
• 🛠️ Tools registered by the plugin

See AUTH_CONFIGURATION.md for authentication setup.

Related Documentation

• DEVELOPER_GUIDE.md - Complete guide for building bot-testable UIs
• AUTH_CONFIGURATION.md - Authentication setup for testing
• @majkapp/plugin-test README - Automated testing framework
• examples/ - Working test examples
• UI_TESTING_DESIGN.md - UI testing architecture
• PLUGIN_INSPECTION_DESIGN.md - Plugin inspection framework

Happy Testing! 🤖

🎯 Using Real Plugin Functions

By default, if you don't provide --handlers, the bot will automatically load your real plugin functions:

npx @majkapp/bot-plugin-server \
  --script ./my-test.js \
  --plugin ./my-plugin

Your plugin's actual functions will be called when the UI invokes them!

📋 Console Log & Error Capture

All client-side JavaScript logs are automatically captured with timestamps and source locations.

Output files in ./logs/:

• console.txt - Human-readable timeline
• console.json - Structured data
• errors.txt - Page errors with stacks
• summary.json - Quick overview

🎮 Interactive Mode

Test manually in a browser while capturing logs:

npx @majkapp/bot-plugin-server \
  --interactive \
  --plugin ./my-plugin

Press Ctrl+C to stop and save all logs.