LocatAI TypeScript

AI-powered element locator for Selenium WebDriver in TypeScript. LocatAI allows you to find web elements using natural language descriptions instead of manually creating complex CSS or XPath selectors.

Features

• 🧠 AI-Powered Location: Use natural language to describe elements instead of writing complex selectors
• 🔀 Multiple AI Providers: Support for OpenAI, Anthropic (Claude), Google Gemini, and Ollama
• 🔄 Smart Retry Logic: Automatically tries multiple locator strategies with confidence scoring
• 💾 Caching: Stores successful locators to speed up repeat lookups
• 📊 Statistics: Tracks locator success rates and timing information
• ⏱️ Smart Waits: Learns how long elements typically take to appear

Installation

npm install locatai-ts

Usage

Basic Example

import { Builder, WebDriver } from 'selenium-webdriver';
import { ElementFinder, AIProviderFactory } from 'locatai-ts';

async function example() {
  // Initialize WebDriver
  const driver = await new Builder().forBrowser('chrome').build();
  
  try {
    // Go to a website
    await driver.get('https://example.com');
    
    // Initialize LocatAI with your preferred AI provider
    const provider = AIProviderFactory.create();
    ElementFinder.initialize(provider);
    
    // Create an instance of ElementFinder
    const finder = new ElementFinder();
    
    // Find elements using natural language descriptions
    const loginButton = await finder.findElementByLocatAI(driver, 'the login button');
    await loginButton.click();
    
    // Find more elements
    const emailField = await finder.findElementByLocatAI(driver, 'email input field');
    await emailField.sendKeys('[email protected]');
    
    // Use with complex descriptions
    const submitButton = await finder.findElementByLocatAI(
      driver, 
      'the blue submit button at the bottom of the form'
    );
    await submitButton.click();
    
  } finally {
    await driver.quit();
  }
}

Configuration

LocatAI can be configured with various options:

import { ElementFinder, AIProviderFactory, LocatAIOptions } from 'locatai-ts';

// Configure LocatAI with custom options
const options: LocatAIOptions = {
  minimumConfidence: 0.7,       // Minimum confidence threshold (0-1)
  maxLocatorsToTry: 5,          // Number of locators to try before giving up
  timeoutMilliseconds: 5000,    // Timeout for finding elements
  enableDetailedLog: true,      // Enable detailed logging
  enableLocatorRetry: true,     // Enable retry logic
  maxRetryAttempts: 3           // Maximum retry attempts
};

// Initialize with options
const provider = AIProviderFactory.create();
ElementFinder.initialize(provider, options);

Using Different AI Providers

LocatAI supports multiple AI providers:

OpenAI (default)

// Set in environment variables:
// API_KEY=your-openai-api-key
// MODEL=gpt-4-turbo-preview (optional)

// Or in appsettings.json:
// {
//   "provider": "openai",
//   "apiKey": "your-api-key",
//   "modelName": "gpt-4-turbo-preview"
// }

Claude by Anthropic

// Set in environment variables:
// LLMAI_PROVIDER=anthropic
// LLMAI_KEY=your-anthropic-api-key
// LLMAI_MODEL=claude-3-opus-20240229 (optional)

// Or directly:
import { AnthropicProvider, ElementFinder } from 'locatai-ts';

const provider = new AnthropicProvider({
  apiKey: 'your-api-key',
  model: 'claude-3-opus-20240229'
});
ElementFinder.initialize(provider);

Google Gemini

// Set in environment variables:
// LLMAI_PROVIDER=gemini
// LLMAI_KEY=your-gemini-api-key
// LLMAI_MODEL=gemini-2.0-flash (optional)

Ollama (local models)

// Set in environment variables:
// LLMAI_PROVIDER=ollama
// LLMAI_KEY=http://localhost:11434 (optional - default URL)
// LLMAI_MODEL=llama2:latest (optional - specify model name)

License

MIT