LocatAI-js

LocatAI is an enterprise-grade AI-powered element locator for Selenium WebDriver. It makes your test automation more reliable by using AI to dynamically locate web elements based on natural language descriptions.

Features

• AI-Powered Element Location: Find elements using natural language descriptions
• Multiple AI Provider Support: Works with OpenAI, Anthropic/Claude, Google Gemini, and Ollama
• Smart Caching: Caches successful locators to improve performance and reduce API costs
• Adaptive Retry Logic: Tries multiple locator strategies with intelligent fallbacks
• WebDriver Extension: Seamlessly extends the Selenium WebDriver with findElementByLocatAI method
• JSON Configuration: Simple setup with appsettings.json
• Cross-Browser Compatibility: Works with all browsers supported by Selenium WebDriver

Installation

npm install locatai-js

Quick Start

Using JSON Configuration (Recommended)

Create an appsettings.json file with your configuration:

{
  "AI": {
    "Provider": "openai",
    "ApiKey": "your-api-key-here",
    "Model": "gpt-4o-mini"
  },
  "Options": {
    "MinimumConfidence": 0.6,
    "MaxLocatorsToTry": 3,
    "TimeoutMilliseconds": 10000,
    "EnableDetailedLogging": false,
    "EnableLocatorRetry": true,
    "MaxRetryAttempts": 2,
    "CacheDirectory": "./LocatAICache"
  }
}

Then in your test file:

const { Builder } = require('selenium-webdriver');
const LocatAI = require('locatai-js');
const path = require('path');

async function runTest() {
  // Create a WebDriver instance
  const driver = await new Builder().forBrowser('chrome').build();
  
  // Initialize LocatAI with your configuration
  LocatAI.initialize(driver, path.join(__dirname, 'appsettings.json'));
  
  try {
    // Navigate to a website
    await driver.get('https://www.saucedemo.com');
    
    // Find elements using natural language descriptions
    const username = await driver.findElementByLocatAI('username input field');
    const password = await driver.findElementByLocatAI('password input field');
    const loginButton = await driver.findElementByLocatAI('login button');
    
    // Interact with elements normally
    await username.sendKeys('standard_user');
    await password.sendKeys('secret_sauce');
    await loginButton.click();
    
    // Find multiple elements
    const inventoryItems = await driver.findElementsByLocatAI('inventory item');
    console.log(`Found ${inventoryItems.length} items`);
  } finally {
    await driver.quit();
  }
}

runTest();

Using Environment Variables

You can also initialize with environment variables:

// Set environment variables
process.env.OPENAI_API_KEY = 'your-openai-api-key';

const { Builder } = require('selenium-webdriver');
const LocatAI = require('locatai-js');

async function runTest() {
  const driver = await new Builder().forBrowser('chrome').build();
  
  // Initialize with a configuration object
  LocatAI.initialize(driver, {
    AI: {
      Provider: 'openai',
      ApiKey: process.env.OPENAI_API_KEY
    },
    Options: {
      EnableDetailedLogging: true
    }
  });
  
  // Now use driver.findElementByLocatAI() as needed
}

AI Provider Configuration

LocatAI-js supports the following AI providers:

| Provider | Configuration Key | Environment Variable | |-----------|-------------------|----------------------| | OpenAI | openai | OPENAI_API_KEY | | Anthropic | anthropic | ANTHROPIC_API_KEY | | Google | gemini | GEMINI_API_KEY | | Ollama | ollama | OLLAMA_HOST |

Advanced Configuration

You can customize the behavior with additional options:

const options = {
  AI: {
    Provider: 'openai',
    ApiKey: 'your-api-key',
    Model: 'gpt-4o' // Specify a specific model
  },
  Options: {
    // Element finder settings
    MinimumConfidence: 0.7,       // Minimum confidence score to accept a locator
    MaxLocatorsToTry: 5,          // Maximum number of locators to try
    
    // Timing settings
    TimeoutMilliseconds: 15000,   // Timeout for finding elements
    
    // Retry settings
    EnableLocatorRetry: true,     // Whether to retry failed locators
    MaxRetryAttempts: 3,          // How many times to retry
    
    // Caching settings
    CacheDirectory: './LocatAICache', // Where to store the cache
    
    // Debug settings
    EnableDetailedLogging: true   // Log detailed information about locators
  }
};

LocatAI.initialize(driver, options);

API Reference

Main Methods

LocatAI.initialize(driver, config)

Initializes LocatAI and extends the WebDriver instance with additional methods.

| Parameter | Type | Description | |-----------|------|-------------| | driver | WebDriver | The Selenium WebDriver instance | | config | Object|String | Configuration object or path to appsettings.json |

driver.findElementByLocatAI(description)

Finds a single element using natural language.

| Parameter | Type | Description | |-----------|------|-------------| | description | String | Natural language description of the element | | Returns | WebElement | The found element |

driver.findElementsByLocatAI(description)

Finds multiple elements using natural language.

| Parameter | Type | Description | |-----------|------|-------------| | description | String | Natural language description of the elements | | Returns | Array | Array of found elements |

Enterprise Support

For enterprise support, custom integration, or training, please contact us at [email protected].

License

MIT

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.