semantic-playwright

Click by description, not selectors.

A Playwright extension for semantic element targeting. Write resilient tests that survive website redesigns.

Installation

npm install semantic-playwright playwright

Quick Start

import { test } from '@playwright/test';
import { semantic } from 'semantic-playwright';

test('login flow', async ({ page }) => {
  await page.goto('https://example.com/login');

  const s = semantic(page);

  // Click and fill by description - no CSS selectors needed
  await s.fill('email input', '[email protected]');
  await s.fill('password field', 'secret123');
  await s.click('sign in button');

  // Wait for success
  await s.waitFor('welcome message');
});

Why?

Modern web UIs change constantly. CSS selectors like #main-nav > ul > li:nth-child(3) > a.btn-primary break the moment a designer moves a button. This library provides self-healing locators that target elements by their semantic purpose.

// Fragile - breaks on redesign
await page.click('#nav-signin-btn');

// Resilient - survives redesign
await s.click('sign in button');

How It Works

The semantic engine classifies DOM elements using multiple signals:

• ARIA roles: role="button", role="navigation"
• Semantic HTML: <nav>, <main>, <button>, <form>
• Text content: "Sign In", "Submit", "Search"
• Class patterns: btn, nav, form-input
• Context: Element inside <nav> likely navigation-related

No LLM required. Pure heuristic matching. Fast, local, privacy-preserving.

API

semantic(page)

Wrap a Playwright page with semantic methods:

const s = semantic(page);

s.click(description, options?)

Click an element by description:

await s.click('submit button');
await s.click('login', { purposeHint: 'action' });

s.fill(description, value, options?)

Fill a form field by description:

await s.fill('email input', '[email protected]');
await s.fill('search box', 'playwright');

s.locator(description, options?)

Get a Playwright Locator for chaining:

const btn = await s.locator('checkout button');
await btn.hover();
await btn.click();

s.analyze(includeUnknown?)

Analyze page structure:

const { elements, summary } = await s.analyze();
console.log(summary);
// { navigation: 12, action: 8, form: 5, content: 20, data: 3, unknown: 0 }

s.query(category, textQuery?)

Find elements by category:

const buttons = await s.query('action');
const searchInputs = await s.query('form', 'search');

s.findAll(description, options?)

Find all matching elements with scores:

const matches = await s.findAll('submit');
for (const { locator, score, text } of matches) {
  console.log(`${text}: ${score}`);
}

s.waitFor(description, options?)

Wait for element to appear:

await s.waitFor('success notification', { timeout: 10000 });

Categories

| Category | Description | Examples | |----------|-------------|----------| | navigation | Navigation elements | Nav bars, menus, breadcrumbs | | action | Interactive actions | Buttons, CTAs, links styled as buttons | | form | Form inputs | Text fields, selects, checkboxes | | content | Main content | Articles, headings, paragraphs | | data | Data display | Tables, lists, grids |

Options

interface SemanticOptions {
  purposeHint?: 'navigation' | 'action' | 'form' | 'content' | 'data';
  timeout?: number;    // Default: 5000ms
  minScore?: number;   // Default: 20
}

Architecture

src/
├── index.ts           # Main API - SemanticPage wrapper
├── semantic-mapper.ts # Classification engine & JS generators

Key design decisions:

• Scripts execute in browser context via page.evaluate()
• Classification is entirely heuristic-based (no external calls)
• Selectors generated are full CSS paths for reliability
• Scoring system prefers exact matches, falls back gracefully

Development

npm install
npm run build
npm test

Testing Philosophy

Tests should use real websites to validate resilience:

1. Test against sites that redesign frequently
2. Snapshot selectors, verify semantic descriptions still work after updates
3. Compare failure rates: CSS selectors vs semantic descriptions

Roadmap

v0.1.0 (Current)

• [x] Core semantic classification engine
• [x] semantic() page wrapper
• [x] click(), fill(), locator() methods
• [x] analyze() and query() methods
• [x] findAll() and waitFor() methods

v0.2.0

• [ ] Playwright Test fixtures (useSemanticPage)
• [ ] Custom matchers (expect(s).toHaveElement('login button'))
• [ ] Confidence thresholds configuration
• [ ] Debug mode with match explanations

v0.3.0

• [ ] Learning/correction persistence
• [ ] Domain-specific overrides
• [ ] Screenshot-on-failure with element highlighting
• [ ] Playwright Reporter integration

v1.0.0

• [ ] Comprehensive test suite
• [ ] Performance benchmarks
• [ ] Documentation site
• [ ] VS Code extension for selector conversion

Prior Art

This library is inspired by and aims to improve upon:

• Browser-Use: AI agent browser automation (requires LLM)
• Healenium: Self-healing Selenium (Java-focused)
• Test.ai: AI-powered testing (SaaS, expensive)

Our differentiator: No LLM required. Fast, local, free.

License

MIT