Probolib: drive your playwright scripts with AI superpowers

Probolib is a powerful AI-driven automation library that enhances Playwright testing and automation by making your scripts more robust and maintainable. Instead of relying on brittle CSS selectors or complex XPath expressions, Probolib uses natural language to interact with web elements.

Why Probolib?

• Natural Language Automation: Write human-readable instructions instead of complex selectors
• More Resilient Tests: AI-powered element detection that adapts to UI changes
• Simpler Maintenance: Reduce the need to update selectors when the UI changes
• Faster Development: Write automation scripts in plain English

Example

Instead of writing:

page.click('some > super > complex > css > and non robust selector')

You can simply write:

probo.runStep(page, 'click on the save button')

Quickstart

npm install @probolabs/playwright

Accessing our backend

the heavy lifting of the AI reasoning is done at the moment on our backend server. in order to access it you would need to tell the library how to access it.

easyiest - set ENV var

probolib expects the existance of 2 env vars:

export PROBO_API_ENDPOINT=api.probolabs.ai
export PROBO_API_KEY=<api key we will give you>

Or use dotenv

// npm install dotenv
// in your script add this line
import 'dotenv/config' // ES6

// .env file located in the root of your project (the same level as your package.json)
PROBO_API_ENDPOINT=https://api.probolabs.ai
PROBO_API_KEY=<api key we will give you>

A complete example for Playwright integration. step by step

step 1: init a playwright project

# from your work directory
mkdir probo-example
cd probo-example
npm init playwright@latest
# follow the instructions and wait till the installation is finished
# verify that playwright is installed properly
npx playwright test --project chromium

step 2: integrate probolabs

npm install @probolabs/playwright dotenv

step 3: configure the env with our endpoint and api key

touch .env

edit the .env file

#.env
PROBO_API_ENDPOINT=https://api.probolabs.ai
PROBO_API_KEY=<api key we will give you>

step 4: create a file under the tests folder named probo-example-todo-mvc.spec.mjs

// tests/probo-example-todo-mvc.spec.mjs
import 'dotenv/config';
import { test} from '@playwright/test';
import { Probo } from '@probolabs/playwright';

//
// Important: Before running this script set PROBO_API_KEY and PROBO_API_ENDPOINT
//

test.describe('Todo MVC', () => {
  test('basic example', async ({ page }) => {
    try {
      // Initialize Probo
      const probo = new Probo({
        scenarioName: 'probo-example-todo-mvc' // important for caching the AI reasoning
      });

      //Goto page
      await page.goto('https://demo.playwright.dev/todomvc');

      // Run test steps
      console.log('Running test steps...');
      await probo.runStep(page, 'enter a new todo item: "Buy groceries"');
      await probo.runStep(page, 'press the Enter key');

      console.log('✨ Test completed successfully!');
    } catch (error) {
      console.error('❌ Test failed:', error);
      throw error; // Re-throw to mark the test as failed
    }
  });
});

run the example

npx playwright test tests/probo-example-todo-mvc.spec.mjs --headed --project chromium

Using Probo Fixtures (Connect to Existing Browser)

Probo fixtures allow you to connect to an existing Chrome instance (like your recorder app) instead of launching a new browser. This is useful when you want to run tests against a browser that's already running with specific extensions or configurations.

Quick Setup

Step 1: Install the package

npm install @probolabs/playwright

Step 2: Create a playwright.config.ts file

// playwright.config.ts
import { defineConfig } from "@playwright/test";

export default defineConfig({
  testDir: "./tests",
  use: {
    // Don't launch a new browser - Probo fixtures will connect to existing one
    launchOptions: {},
  },
  workers: 1, // Use 1 worker to avoid conflicts with existing browser
});

Step 3: Use Probo fixtures in your test files

// tests/example.spec.ts
import { test, expect } from "@probolabs/playwright/fixtures";

test.describe("My Tests", () => {
  test("should work with existing browser", async ({ page }) => {
    // This page comes from your existing Chrome instance
    await page.goto("https://example.com");
    await page.click("button");

    // Your test code here...
  });
});

Step 4: Make sure your recorder app is running, then run tests

npx playwright test

Advanced Configuration

You can customize the fixtures with different options:

// tests/example.spec.ts
import { createProboFixtures, expect } from "@probolabs/playwright/fixtures";

// Create custom fixtures with specific configuration
const test = createProboFixtures({
  debugPort: 9333, // Default port where your recorder app runs
  showBrowserConsole: true, // Show browser console logs
});

test.describe("My Tests", () => {
  test("custom configuration", async ({ page }) => {
    // Your test code here...
  });
});

Requirements

• Your recorder app must be running with Chrome accessible via CDP
• Chrome must be launched with --remote-debugging-port=9333 (or your custom port)
• The browser must have at least one page open (not just extension pages)

Troubleshooting

Error: "No browser context found"

• Make sure your recorder app is running
• Check that Chrome is accessible at http://localhost:9333
• Verify the debug port matches your configuration

Error: "No main page found"

• Make sure your recorder app has opened a page
• The page should not be a chrome-extension:// URL

Mailpit OTP Integration (Mailinator Replacement)

The OTP utilities now use Mailpit by default. Configuration happens via environment variables:

# defaults shown below
export MAILPIT_BASE_URL="http://mailpit.probolabs.ai:8025"
export MAILPIT_DEFAULT_DOMAIN="mailpit.probolabs.ai"
export MAILPIT_USERNAME=""   # optional (basic auth)
export MAILPIT_PASSWORD=""   # optional (basic auth)
export MAILPIT_FETCH_LIMIT=50 # optional (message batch size)

Key points:

• Leave the env vars unset to rely on the shared mailpit.probolabs.ai instance.
• Set MAILPIT_USERNAME/MAILPIT_PASSWORD only if your Mailpit instance is protected with HTTP basic auth.
• When the recorder app sends an email, OTP.waitForOTP() can poll all inboxes or a specific address ({ inbox: '[email protected]' }).
• Messages are removed via DELETE /api/v1/messages after a code is captured. Disable that behavior by toggling OTP.DELETE_MESSAGE_AFTER_OTP_EXTRACTION.

Tests in packages/probo-playwright/test (wait-for-otp.js, extract-otp-from-message.js) are wired to the same defaults, so they should work out of the box.