@x360airuntest/agent

AI-powered browser test automation. Describe what to do — the agent handles the rest.

npm install @x360airuntest/agent

Overview

@x360airuntest/agent is an AI-driven browser automation agent built on patchright (anti-detection Playwright fork). You write test intent in plain English; the agent navigates, interacts with, and verifies your web app autonomously. Each step is classified by an LLM judge as Passed, AI Smart Passed (goal met via an alternate path), or Failed, and a full execution report is generated.

Capabilities

| Feature | Description | |---|---| | Natural-language tasks | Describe goals — no selectors or locators needed | | Multi-LLM | OpenAI, Azure OpenAI, Anthropic, Gemini, DeepSeek | | Self-healing | Automatically adapts strategy when UI changes break a step | | Step evaluation | LLM judge classifies each step with intermittent stability checks | | Network monitoring | Capture/filter API traffic during execution | | Video recording | Per-session video saved to disk | | CDP actions | Low-level Chrome DevTools Protocol interactions | | MCP integration | Connect external tool servers (stdio / SSE) | | Faker data generation | Auto-generate realistic test data for form fields | | Execution reports | Structured JSON history and step-by-step classification |

Quick start

Prerequisites

# Install patchright browsers (run once after install)
npx patchright install chromium

Minimal example

import { X360AIAgent } from "@x360airuntest/agent";

const agent = new X360AIAgent({
  llm: { provider: "openai", model: "gpt-4o", apiKey: process.env.OPENAI_API_KEY },
  apiKey: process.env.X360_AUTOMATION_API_KEY,
  accountId: process.env.X360_AUTOMATION_ACCOUNT_ID,
  projectId: process.env.X360_AUTOMATION_PROJECT_ID,
});

const result = await agent.runTest("User registration", [
  "Go to https://app.acme.io/signup",
  "Fill in the first name field with 'Sarah'",
  "Fill in the last name field with 'Connor'",
  "Enter [email protected] in the email field",
  "Set a strong password and confirm it",
  "Click the Create Account button",
  "Verify a welcome email confirmation message is shown",
]);

console.log(result.status); // "Passed" | "AI Smart Passed" | "Failed"
await agent.close();

Class

X360AIAgent

The single unified agent class for all automation and test use cases. Supports both freeform task execution and structured test-case execution with per-step LLM evaluation.

The core automation agent. Use this for freeform task execution, scripting, or building workflows that go beyond structured test cases.

import { X360AIAgent } from "@x360airuntest/agent";

// Freeform task execution
const agent = new X360AIAgent({
  llm: {
    provider: "azure-openai",
    model: "gpt-4o",
    endpoint: process.env.AZURE_ENDPOINT,
    deployment: "gpt-4o",
    apiVersion: "2024-02-01",
  },
  apiKey: process.env.X360_AUTOMATION_API_KEY,
  accountId: process.env.X360_AUTOMATION_ACCOUNT_ID,
  projectId: process.env.X360_AUTOMATION_PROJECT_ID,
  debug: true,
  recordVideo: true,
});

const result = await agent.run(
  "Go to https://admin.acme.io, sign in with the credentials in the environment, " +
  "navigate to the Billing section, and return the current plan name and renewal date"
);
console.log(result.output); // e.g. "Pro Plan – renews 2026-07-01"
await agent.close();

// Structured test-case execution
const agent = new X360AIAgent({
  llm: { provider: "openai", model: "gpt-4o", apiKey: process.env.OPENAI_API_KEY },
  apiKey: process.env.X360_AUTOMATION_API_KEY,
  accountId: process.env.X360_AUTOMATION_ACCOUNT_ID,
  projectId: process.env.X360_AUTOMATION_PROJECT_ID,
  enableEvaluation: true,
  enableSelfHealing: true,
});

const result = await agent.runTest("Password reset flow", [
  "Navigate to https://app.acme.io/forgot-password",
  "Enter the email address [email protected]",
  "Click the Send Reset Link button",
  "Verify a confirmation message saying 'Check your inbox' is displayed",
  "Confirm no error messages are present on the page",
]);

console.log(result.testCaseName, result.status);
await agent.close();

Agent methods

| Method | Description | |---|---| | run(task, params?) | Execute a freeform task and wait for completion | | runAsync(task, params?) | Execute a task asynchronously, returns a Task control handle | | runTest(name, steps, params?) | Run a named test case with per-step evaluation | | evaluate(objective, steps, status, originalSteps?) | Evaluate a completed task against expected steps | | evaluateActions(objective, steps) | Evaluate a sequence of page.aiAction() calls | | evaluateStep(step, executed, index, ...) | Evaluate one specific step in isolation | | clearEvaluationCache() | Clear cached LLM evaluation results | | connectMCP(config) | Connect MCP tool servers at startup | | addMCPServer(config) | Connect an additional MCP server at runtime | | removeMCPServer(serverId) | Disconnect a specific MCP server | | listMCPServers() | Get IDs of all connected MCP servers | | getMCPInfo() | Get tool counts and names per MCP server | | newPage() | Create a new browser page | | getPages() | Get all open pages | | getActivePage() | Get or create the current active page | | launchBrowser() | Explicitly launch the browser | | addVariable(variable) | Add a named variable for use in tasks | | getVariable(key) | Get a stored variable | | deleteVariable(key) | Delete a stored variable | | getVariables() | Get all stored variables | | getActionHistory() | Get tracked aiAction() call history | | clearActionHistory() | Clear tracked action history | | getNetworkData() | Get captured network calls (if monitoring enabled) | | close() | Close the agent and all browser resources |

Configuration — X360AIAgentConfig

const agent = new X360AIAgent({
  // -- LLM (required unless OPENAI_API_KEY is set) -----------------------
  llm: { provider: "openai", model: "gpt-4o", apiKey: "..." },

  // -- Automation API credentials (required for all AI methods) ---------
  apiKey:     process.env.X360_AUTOMATION_API_KEY,
  accountId:  process.env.X360_AUTOMATION_ACCOUNT_ID,
  projectId:  process.env.X360_AUTOMATION_PROJECT_ID,

  // -- Browser ----------------------------------------------------------
  browserProvider: "Playwright",          // "Playwright" (default)
  browserOptions: {
    viewport: { width: 1920, height: 1080 },
    proxy: { server: "http://proxy:8080" },
    deviceScaleFactor: 2,
  },
  recordVideo: true,                      // saves to videos/ by default
  // recordVideo: { dir: "./recordings", size: { width: 1280, height: 720 } },

  // -- Evaluation -------------------------------------------------------
  enableEvaluation: true,
  enableSelfHealing: true,

  // -- Network monitoring -----------------------------------------------
  enableNetworkMonitoring: true,
  networkMonitorOptions: {
    mode: "all",                          // "all" | "action-specific"
    urlFilters: ["/api/"],
    excludePatterns: ["google-analytics", "doubleclick"],
  },

  // -- Advanced ---------------------------------------------------------
  debug: false,
  enableCDP: true,
  actions: [],                            // custom browser interactions
  timing: { navigationTimeoutMs: 60000, typeDelayMs: 50 },
  debugFlags: {
    cdpSessions: false,
    traceWait: false,
    profileDomCapture: false,
    structuredSchema: false,
  },
});

LLM providers

Pass an LLMConfig object to llm:, or construct a client with createLLMClient().

// OpenAI
{ provider: "openai", model: "gpt-4o", apiKey: process.env.OPENAI_API_KEY }

// Azure OpenAI
{
  provider: "azure-openai",
  model: "gpt-4o",
  endpoint: "https://my-resource.openai.azure.com",
  deployment: "gpt-4o",
  apiVersion: "2024-02-01",
}

// Anthropic
{ provider: "anthropic", model: "claude-3-5-sonnet-20241022", apiKey: process.env.ANTHROPIC_API_KEY }

// Gemini
{ provider: "gemini", model: "gemini-1.5-pro", apiKey: process.env.GEMINI_API_KEY }

// DeepSeek
{ provider: "deepseek", model: "deepseek-chat", apiKey: process.env.DEEPSEEK_API_KEY }

To bring your own LLM client, implement the LocalBrowserLLM interface and pass the instance directly.

Page-level AI methods

Once a page is acquired via agent.getActivePage(), agent.newPage(), or agent.getPages(), it exposes AI-powered methods:

const page = await agent.getActivePage();
await page.goto("https://app.acme.io/dashboard");

// Run a multi-step freeform task on this page
const result = await page.ai(
  "Find the latest invoice in the Billing table and open its details"
);

// Execute a single granular action (1 LLM call, no agent loop)
await page.aiAction("Click the Export CSV button");
await page.aiAction("Select 'Last 30 days' from the date range dropdown");
await page.aiAction("Type 'Q2 Report' into the filename field");

// Run a task asynchronously — continue other work while it runs
const task = await page.aiAsync("Fill in all required fields in the onboarding wizard");

// Extract structured data from the page
const z = require("zod");
const invoiceSchema = z.object({
  invoiceNumber: z.string(),
  amount: z.number(),
  status: z.enum(["paid", "pending", "overdue"]),
});
const invoice = await page.extract("Get the invoice number, amount, and status", invoiceSchema);
console.log(invoice); // { invoiceNumber: "INV-0042", amount: 299, status: "paid" }

Step evaluation

Every executed step is classified by an independent LLM judge:

| Status | Meaning | |---|---| | Passed | Agent fully accomplished exactly what the step required | | AI Smart Passed | Goal achieved via an alternate path or phrasing (same intent met) | | Failed | Step was not accomplished |

Self-healing

When enableSelfHealing: true, the agent monitors for UI and API failures after each action and retries with an adapted strategy automatically.

const agent = new X360AIAgent({
  llm: { provider: "anthropic", model: "claude-3-5-sonnet-20241022", apiKey: process.env.ANTHROPIC_API_KEY },
  apiKey: process.env.X360_AUTOMATION_API_KEY,
  accountId: process.env.X360_AUTOMATION_ACCOUNT_ID,
  projectId: process.env.X360_AUTOMATION_PROJECT_ID,
  enableSelfHealing: true,
  enableEvaluation: true,
});

// If a button moves, gets renamed, or is temporarily disabled,
// the agent automatically tries alternative approaches before failing.
const result = await agent.runTest("Add team member", [
  "Go to https://app.acme.io/settings/team",
  "Click the Invite Member button",
  "Enter [email protected] in the email field",
  "Select the 'Editor' role",
  "Confirm the invitation was sent successfully",
]);
console.log(result.status); // self-healed steps are still reported

For fine-grained control in custom workflows, use SelfHealingExecutor directly:

import { SelfHealingExecutor, AdaptationStrategy } from "@x360airuntest/agent";

Faker data generation

The agent can automatically generate realistic test data for form fields. The utilities are available directly:

import {
  detectFieldType,
  generateFakerValue,
  autoFakerValue,
  FAKER_EXPRESSION_MAP,
} from "@x360airuntest/agent";

// Detect what kind of field an input is
detectFieldType("billing address");     // "address"
detectFieldType("mobile number");       // "mobile"
detectFieldType("date of birth");       // "birthdate"
detectFieldType("company name");        // "company"

// Generate a runtime value for a specific field type
generateFakerValue("email");            // "[email protected]"
generateFakerValue("phone");            // "+1-555-203-4891"
generateFakerValue("zipCode");          // "90210"
generateFakerValue("amount");           // "74.50"

// Auto-detect the field type from a label and generate in one call
autoFakerValue("Full legal name");      // "James Thornton"
autoFakerValue("Street address");       // "742 Evergreen Terrace"
autoFakerValue("Account number");       // "4532015112830366"

// Retrieve the faker.js expression for Playwright script generation
FAKER_EXPRESSION_MAP["email"];          // "faker.internet.email()"
FAKER_EXPRESSION_MAP["birthdate"];      // "faker.date.birthdate()"
FAKER_EXPRESSION_MAP["uuid"];           // "faker.string.uuid()"

Custom browser interactions

Extend the agent with your own interaction types:

import { X360AIAgent } from "@x360airuntest/agent";
import type { BrowserInteractionDefinition } from "@x360airuntest/agent/types";
import { z } from "zod";

// Custom action: dismiss cookie consent banner before any test step
const dismissCookieBanner: BrowserInteractionDefinition = {
  type: "dismiss_cookie_banner",
  description: "Dismiss the cookie consent / GDPR banner if it is visible",
  schema: z.object({}),
  execute: async ({ page }) => {
    const selectors = ["[data-testid='cookie-accept']", "#onetrust-accept-btn-handler", ".cc-accept"];
    for (const sel of selectors) {
      const btn = page.locator(sel).first();
      if (await btn.isVisible().catch(() => false)) {
        await btn.click();
        return { success: true, message: `Cookie banner dismissed via ${sel}` };
      }
    }
    return { success: true, message: "No cookie banner found" };
  },
};

// Custom action: take a labelled screenshot at any point in a run
const screenshotAction: BrowserInteractionDefinition = {
  type: "take_screenshot",
  description: "Capture a screenshot with a descriptive label",
  schema: z.object({ label: z.string() }),
  execute: async ({ page, params }) => {
    const path = `screenshots/${params.label.replace(/\s+/g, "-")}-${Date.now()}.png`;
    await page.screenshot({ path, fullPage: true });
    return { success: true, message: `Screenshot saved to ${path}` };
  },
};

const agent = new X360AIAgent({
  llm: { provider: "openai", model: "gpt-4o" },
  apiKey: process.env.X360_AUTOMATION_API_KEY,
  accountId: process.env.X360_AUTOMATION_ACCOUNT_ID,
  projectId: process.env.X360_AUTOMATION_PROJECT_ID,
  actions: [dismissCookieBanner, screenshotAction],
});

MCP tool servers

Connect Model Context Protocol (MCP) servers to give the agent additional tools:

await agent.connectMCP({
  servers: [
    {
      // Read/write test fixtures on disk
      id: "fixtures",
      connectionType: "stdio",
      command: "npx",
      args: ["-y", "@modelcontextprotocol/server-filesystem", "./test-fixtures"],
    },
    {
      // Internal REST API tool server (seed data, reset state, etc.)
      id: "test-api",
      connectionType: "sse",
      sseUrl: "http://localhost:4200/mcp/sse",
      sseHeaders: { Authorization: `Bearer ${process.env.TEST_API_TOKEN}` },
    },
  ],
});

// Attach an additional database-seeding server mid-run
const dbServerId = await agent.addMCPServer({
  id: "db-seeder",
  connectionType: "stdio",
  command: "node",
  args: ["./tools/db-seeder-mcp.js"],
});

console.log(agent.listMCPServers()); // ["fixtures", "test-api", "db-seeder"]
console.log(agent.getMCPInfo());     // [{ id, toolCount, toolNames }, ...]

// Disconnect after seeding is done
agent.removeMCPServer(dbServerId);

Environment variables

| Variable | Purpose | |---|---| | OPENAI_API_KEY | Auto-configures Azure OpenAI gpt-4o when no llm option is provided | | X360_AUTOMATION_API_KEY | Required for all AI methods (agent.run(), page.ai(), page.aiAction(), page.extract()) | | X360_AUTOMATION_ACCOUNT_ID | Required alongside the automation API key | | X360_AUTOMATION_PROJECT_ID | Required alongside the automation API key | | X360_ENVIRONMENT | development | staging | production (default: production) |

Full export reference

import {
  // -- Agent -------------------------------------------------------------
  X360AIAgent,            // unified automation + test agent
  X360AITestAgent,        // backward-compatible alias for X360AIAgent

  // -- Evaluation system -------------------------------------------------
  StepEvaluator,
  StepStatus,
  SelfHealingExecutor,
  AdaptationStrategy,
  FinalEvaluator,
  TestEvaluationAgent,

  // -- History -----------------------------------------------------------
  AgentHistoryList,

  // -- Types -------------------------------------------------------------
  TaskStatus,

  // -- Timing (advanced override — smart defaults used automatically) -----
  setTimingConfig,
  getTimingConfig,

  // -- Faker data generation ---------------------------------------------
  detectFieldType,
  generateFakerValue,
  autoFakerValue,
  isHardcodedInInstruction,
  FAKER_EXPRESSION_MAP,

  // -- Browser interactions ----------------------------------------------
  SwitchTabInteractionDefinition,
} from "@x360airuntest/agent";

// Path-scoped imports
import type { X360AIAgentConfig, BrowserProviders } from "@x360airuntest/agent/types";
import type { BrowserInteractionDefinition, TimingConfig, FakerFieldType } from "@x360airuntest/agent/types";
import type { EvaluationResult } from "@x360airuntest/agent";

License

Proprietary — commercial use requires a license agreement with X360 AI Tech. Contact [email protected].