playwright-genie

Find and interact with Playwright elements using natural language — powered by any LLM

playwright-genie lets you write Playwright tests in plain English. No more hunting for selectors — just describe the element and the genie finds it.

Features

• Natural Language — describe elements in plain English, no selectors needed
• 40+ Playwright Actions — click, fill, check, hover, drag, wait, screenshot and more
• Any LLM Provider — OpenAI, Claude, Ollama, Azure, or any OpenAI-compatible API
• Smart Caching — persistent disk cache + in-memory cache to minimize LLM calls
• Action-Aware — fill('username') targets the input, not the label
• Auto-Retry — stale cached locators are automatically re-resolved
• TypeScript Support — full type definitions included
• Single Page Object — one createSmartLocator(page) works across all navigations

Installation

npm install playwright-genie

Prerequisites

• Node.js >= 18
• Playwright >= 1.40
• An LLM API key (OpenAI, Anthropic, or any OpenAI-compatible provider)

Setup

Create a .env file in your project root:

# Option 1: OpenAI
LLM_API_KEY=sk-your-openai-key
LLM_MODEL=gpt-4o-mini

# Option 2: Anthropic (via OpenAI-compatible proxy)
LLM_API_KEY=your-anthropic-key
LLM_BASE_URL=https://your-proxy.com/v1
LLM_MODEL=claude-sonnet-4-20250514

# Option 3: Ollama (local, free)
LLM_API_KEY=ollama
LLM_BASE_URL=http://localhost:11434/v1
LLM_MODEL=llama3

# Option 4: Azure OpenAI
LLM_API_KEY=your-azure-key
LLM_BASE_URL=https://your-resource.openai.azure.com/openai/deployments/your-deployment
LLM_MODEL=gpt-4o-mini

Also supports OPENAI_API_KEY, ANTHROPIC_API_KEY, or ROUTELLM_API_KEY as fallbacks.

Quick Start

With Playwright Test

import { test } from '@playwright/test';
import { createSmartLocator } from 'playwright-genie';

test('login flow', async ({ page }) => {
  const smart = createSmartLocator(page);

  await page.goto('https://myapp.com/login');
  await smart.fill('username', 'admin');
  await smart.fill('password', 'secret123');
  await smart.click('login button');
  await smart.waitForVisible('welcome heading');
});

Standalone Script

import { chromium } from 'playwright';
import { createSmartLocator } from 'playwright-genie';

const browser = await chromium.launch();
const page = await browser.newPage();
const smart = createSmartLocator(page);

await page.goto('https://myapp.com');
await smart.click('sign in link');
await smart.fill('email field', '[email protected]');
await smart.fill('password field', 'secret');
await smart.click('submit button');

await browser.close();

API Reference

createSmartLocator(page, options?)

Creates a smart locator instance bound to a Playwright page. Works across navigations — no need to recreate it.

const smart = createSmartLocator(page, { verbose: true });

Options:

| Option | Type | Default | Description | |--------|------|---------|-------------| | verbose | boolean | false | Log resolved locators to console | | debug | boolean | false | Enable detailed debug logging | | model | string | env var | Override LLM model | | temperature | number | 0 | LLM temperature | | maxTokens | number | 1024 | Max response tokens |

Interaction Actions

await smart.click('login button');
await smart.click('submit', { force: true });

await smart.dblclick('editable cell');

await smart.fill('username', 'Admin');
await smart.fill('email field', '[email protected]', { timeout: 5000 });

await smart.type('search box', 'hello');

await smart.pressSequentially('otp input', '123456', { delay: 100 });

await smart.press('search box', 'Enter');

await smart.clear('email field');

await smart.hover('profile menu');

await smart.focus('first input');

await smart.tap('mobile menu icon');

await smart.select('country dropdown', 'India');

await smart.selectText('paragraph content');

Checkbox & Radio

await smart.check('remember me checkbox');

await smart.uncheck('newsletter opt-in');

await smart.setChecked('terms checkbox', true);

File Upload

await smart.setInputFiles('file upload', '/path/to/file.pdf');
await smart.setInputFiles('avatar input', ['/img1.png', '/img2.png']);

Drag & Drop

const { source, target } = await smart.dragTo('card item', 'drop zone');

Wait Actions

await smart.waitForVisible('success toast');
await smart.waitForVisible('modal', 10000);

await smart.waitForHidden('loading spinner');

await smart.waitForAttached('dynamic table');

await smart.waitForDetached('old modal');

await smart.waitFor('element', { state: 'visible', timeout: 5000 });

State Queries

const visible  = await smart.isVisible('error message');
const hidden   = await smart.isHidden('loading spinner');
const enabled  = await smart.isEnabled('submit button');
const disabled = await smart.isDisabled('locked field');
const checked  = await smart.isChecked('terms checkbox');
const editable = await smart.isEditable('readonly field');
const found    = await smart.exists('optional element');

Content Retrieval

const text  = await smart.getText('welcome heading');
const inner = await smart.getInnerText('article body');
const html  = await smart.getInnerHTML('rich content area');
const value = await smart.getInputValue('email field');
const attr  = await smart.getAttribute('link', 'href');
const box   = await smart.getBoundingBox('hero image');
const num   = await smart.count('list items');

Scroll & Visual

await smart.scrollIntoView('footer section');

const buffer = await smart.screenshot('chart area', { path: 'chart.png' });

await smart.highlight('target element');

smart.locate() — Resolve Once, Act Many Times

When you need multiple actions on the same element, use locate() to resolve the locator once:

const el = await smart.locate('username');
await el.clear();
await el.fill('NewAdmin');
await el.press('Tab');
console.log(await el.inputValue());
console.log(await el.isEnabled());

// Access the raw Playwright locator
const loc = el.rawLocator;

// SmartAction has 40+ methods matching Playwright's Locator API
await el.click();
await el.hover();
await el.screenshot({ path: 'element.png' });
await el.waitForVisible();
await el.evaluate((node) => node.style.border = '2px solid red');

smart.prefetch() — Batch Resolve

Pre-resolve multiple locators in a single LLM call to save time and cost:

await smart.prefetch('username', 'password', 'login button');

// These now hit the cache — no LLM calls
await smart.fill('username', 'Admin');
await smart.fill('password', 'secret');
await smart.click('login button');

Cache Management

smart.clearCache();       // clear in-memory cache only
smart.clearAllCaches();   // clear both memory + disk (.locator-cache.json)

Caching

playwright-genie uses a two-level cache to minimize LLM calls:

1. Memory cache — instant lookups within the same test run
2. Disk cache (.locator-cache.json) — persists across runs

Cache keys are scoped by URL pathname + action + query, so fill('username') on /login won't collide with click('username') on /dashboard.

If a cached locator becomes stale (element no longer exists), it's automatically invalidated and re-resolved via LLM.

Set LOCATOR_CACHE_FILE env var to customize the cache file path.

LLM Provider Configuration

| Provider | LLM_API_KEY | LLM_BASE_URL | LLM_MODEL | |----------|---------------|-----------------|-------------| | OpenAI | sk-... | (default) | gpt-4o-mini | | Anthropic | sk-ant-... | proxy URL | claude-sonnet-4-20250514 | | Ollama | ollama | http://localhost:11434/v1 | llama3 | | Azure OpenAI | Azure key | deployment URL | gpt-4o-mini | | RouteLLM | key | proxy URL | model name |

Complete Examples

Login Flow

import { test, expect } from '@playwright/test';
import { createSmartLocator } from 'playwright-genie';

test('complete login flow', async ({ page }) => {
  const smart = createSmartLocator(page);
  await page.goto('https://myapp.com/login');

  await smart.fill('username', 'admin');
  await smart.fill('password', 'secret123');

  if (await smart.exists('remember me checkbox')) {
    await smart.check('remember me checkbox');
  }

  await smart.click('sign in button');
  await smart.waitForVisible('dashboard heading');

  const welcome = await smart.getText('welcome message');
  expect(welcome).toContain('admin');
});

E-commerce Flow

test('shopping flow', async ({ page }) => {
  const smart = createSmartLocator(page);
  await page.goto('https://shop.example.com');

  await smart.fill('search bar', 'wireless headphones');
  await smart.press('search bar', 'Enter');
  await smart.waitForVisible('product list');

  await smart.click('first product card');
  await smart.select('size dropdown', 'Medium');
  await smart.click('add to cart button');

  await smart.waitForVisible('cart badge');
  const count = await smart.getText('cart badge');
  expect(count).toBe('1');
});

Dynamic Content & Modals

test('handle dynamic content', async ({ page }) => {
  const smart = createSmartLocator(page);
  await page.goto('https://app.example.com');

  if (await smart.exists('cookie consent popup')) {
    await smart.click('accept cookies button');
    await smart.waitForHidden('cookie consent popup');
  }

  await smart.scrollIntoView('footer section');
  await smart.waitForVisible('load more button');
  await smart.click('load more button');
  await smart.waitForHidden('loading spinner');
});

Form Validation

test('form validation', async ({ page }) => {
  const smart = createSmartLocator(page);
  await page.goto('https://myapp.com/signup');

  await smart.click('submit button');
  await smart.waitForVisible('email error message');

  const error = await smart.getText('email error message');
  expect(error).toContain('required');

  await smart.fill('email field', '[email protected]');
  const enabled = await smart.isEnabled('submit button');
  expect(enabled).toBe(true);
});

Drag & Drop

test('kanban board', async ({ page }) => {
  const smart = createSmartLocator(page);
  await page.goto('https://app.example.com/board');

  await smart.dragTo('first task card', 'done column');
  await smart.waitForVisible('task moved toast');
});

Best Practices

Be specific about element type:

await smart.click('login button');        // good
await smart.click('button');              // too vague

Include context when needed:

await smart.click('delete button in first row');
await smart.fill('search box in header', 'shoes');

Use action-appropriate descriptions:

await smart.fill('username', 'Admin');    // finds the input
await smart.click('username label');       // finds the label

Reuse the same instance across navigations:

const smart = createSmartLocator(page);
await page.goto('/login');
await smart.fill('username', 'Admin');
await smart.click('login button');
// navigated to /dashboard — same smart object works
await smart.click('settings tab');

Use locate() for multiple actions on same element:

const el = await smart.locate('search box');
await el.fill('query');
await el.press('Enter');
// 1 LLM call instead of 2

TypeScript

import { test } from '@playwright/test';
import { createSmartLocator, SmartAction, SmartLocator } from 'playwright-genie';

test('typed example', async ({ page }) => {
  const smart: SmartLocator = createSmartLocator(page);

  const el: SmartAction = await smart.locate('username');
  await el.fill('Admin');

  const visible: boolean = await smart.isVisible('dashboard');
  const text: string | null = await smart.getText('heading');
});

Debug Mode

LLM_LOCATOR_DEBUG=true npx playwright test

This logs:

• LLM queries and responses
• Cache hits/misses (memory and disk)
• Page structure payload sizes
• Stale cache invalidations

Security Notes

• The library sends the page's accessibility tree to your configured LLM API
• Sensitive data visible in the DOM may be sent to the API
• Use environment variables for API keys — never hardcode them
• For sensitive environments, use a local LLM (e.g., Ollama)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License — see the LICENSE file for details.

Acknowledgments

• Playwright — browser automation framework
• OpenAI — LLM API support
• Claude Code — AI-powered code generation and architecture design