Artes is a test runner for Playwright that executes predefined Cucumber tests and can generate Allure reports for test results. It simplifies setting up Playwright with Cucumber in your automation workflow. With Artes, you can easily run tests without writing step definitions, generate reports, and customize your testing environment.

🌟 Artes Benefits

🚀 Fast Setup & Smooth Onboarding

• Install in minutes and create a test project effortlessly
• Well-structured, easy-to-follow documentation for a smooth learning curve
• Designed for long-term maintainability and scalability

🧩 Powerful & QA-Friendly Architecture

• Intuitive API for writing custom step definitions
• Rich set of ready-to-use step definitions to speed up test creation
• Fully extensible — add your own step definitions anytime

🌐 Advanced API Testing

• Schema validation to ensure API response correctness
• Rich assertion library for precise validations
• Support for all HTTP methods
• Environment-based API configuration for flexible testing

🔁 Smart Variable & Data Handling

• Page Object Model (POM) support for structured data management
• Save, reuse, and share variables across steps
• Built-in data randomization for dynamic and realistic test data
• Environment-specific variables for clean environment separation

🖥️ Modern UI Automation

• Wide locator strategy support (CSS, XPath, text-based, and more)
• Built-in browser actions
• Cookie management
• Local & session storage handling

⚙️ Flexible Configuration & Hooks

• Environment-based configuration system
• Powerful and customizable configuration files
• Full hook support:
  • Before / After
  • Step-level and scenario-level hooks

🧪 CLI, CI/CD & Containerization

• Powerful CLI for full control from the command line
• Quality gate support — set a minimum success rate threshold to automatically fail the pipeline when test results drop below the required percentage
• Official Artes Docker image for seamless containerized execution
• CI/CD-ready — integrate easily with any pipeline

🤖 AI-Powered Bug Reports & Test Summaries

• Automatically generates professional bug reports for failed test cases
• Generates concise test summaries for passed scenarios
• Supports multiple AI providers — OpenAI, Google Gemini, Anthropic Claude, Mistral, Groq, Cohere, DeepSeek (see Supported AI Providers for full model list and setup)
• Works with local AI models (Ollama, LM Studio) — no API key required
• Multi-language report generation — produce reports in any language
• Configurable report cap to control API usage and costs

📊 Artes Reporting System

• Easy installation with docker compose (For detailed info: Artes Reporting System)
• Multiple reporting formats supported
• Native Allure reporting integration
• Customizable Artes Reporting System

🧑‍💻 Installation

You can install Artes via npm. To install it globally (RECOMMENDED), run the following command:

npm install -g artes

To install it locally in your project, run:

npm install artes

Once installed, you can run Artes using:

npx artes [options]

💡 Usage

Artes has following CLI options:

npx artes [options]

General ( artes -h or artes --help )

| Option | Description | Usage Example | | --- | --- | --- | | -h, --help | Show this help message | artes -h or artes --help | | -v, --version | Show current version of artes | artes -v or artes --version | | -c, --create | Create example artes project | artes -c or artes --create | | -y, --yes | Skip confirmation prompt when creating a project | artes -c -y or artes -c --yes | | --noDeps | Create project without installing dependencies | artes -c --noDeps |

Reporting & Branding ( artes report --help )

| Option | Description | Usage Example | | --- | --- | --- | | -r, --report | Run tests and generate Allure report | artes -r or artes --report | | --reportSuccess | Generate screenshot and video with successful tests too | artes --reportSuccess | | --trace | Enable tracing for all tests | artes --trace | | -rwt, --reportWithTrace | Include trace in the report | artes -rwt or artes --reportWithTrace | | --singleFileReport | Generate single file Allure report | artes -r --singleFileReport | | --zip | Zip the report folder after generation | artes -r --zip | | --logo | Set a custom logo in the report sidebar | artes --logo logo.png | | --brandName | Set the brand name displayed next to the logo | artes --brandName 'My Company' | | --reportName | Report name on the summary widget | artes --reportName 'Alma UI' | | --uploadReport | Upload the generated report to Artes Reporting System | artes -r --zip --uploadReport --reporterURL "https://example.com" | | --reporterURL | URL of the Artes Reporting System | artes --uploadReport --reporterURL "https://example.com" | | --projectName | Project name in the Artes Reporting System (default: "Artes Report") | artes --uploadReport --reporterURL "https://example.com" --projectName "My Project" | | --projectType | Project type for reporting, e.g., UI, API (default: "Artes") | artes --uploadReport --reporterURL "https://example.com" --projectType "API" | | --reportPath | Path to the report zip file to upload (default: ./report.zip) | artes --uploadReport --reporterURL "https://example.com" --reportPath "./my_report.zip" |

Browser ( artes browser --help )

| Option | Description | Usage Example | | --- | --- | --- | | --browser | Specify browser to use (chromium, firefox, webkit) | artes --browser chromium | | --device | Emulate a specific device (e.g., "iPhone 13") | artes --device "iPhone 13" | | --maxScreen | Maximize browser window on launch | artes --maxScreen | | --width | Set browser width (default: 1280) | artes --width 1920 | | --height | Set browser height (default: 720) | artes --height 1080 |

Execution ( artes execution --help )

| Option | Description | Usage Example | | --- | --- | --- | | --headless | Run browser in headless mode | artes --headless | | --baseURL | Set base URL for the tests | artes --baseURL "https://example.com" | | --offline | Run browser in offline mode | artes --offline | | --features | Specify feature file paths to run (comma-separated) | artes --features "tests/features/Alma,tests/features/Banan.feature" | | --stepDef | Specify step definition file paths to use (comma-separated) | artes --stepDef "tests/steps/login.js,tests/steps/home.js" | | --tags | Run tests with specified Cucumber tags | artes --tags "@smoke and not @wip" | | --env | Set the environment for the test run | artes --env "dev" | | --saveVar | Set variables from CLI | artes --saveVar '{"armud":20,"banana":200}' | | --parallel | Run tests in parallel mode | artes --parallel 3 | | --retry | Retry failed tests | artes --retry 2 | | --rerun | Rerun only failed tests from previous run | artes --rerun @rerun.txt | | --dryRun | Perform a dry run without executing tests | artes --dryRun | | --percentage | Set minimum success percentage to pass test run (default: 0) | artes --percentage 85 | | --timeout | Set timeout for each test step in seconds (default: 30) | artes --timeout 10 | | --slowMo | Slow down execution for clear view (default: 0) | artes --slowMo 1 |

AI Bug Reporter ( artes ai --help )

| Option | Description | Usage Example | | --- | --- | --- | | --ai | Enable AI-generated bug reports and test summaries | artes --ai | | --aiModel | AI model to use for report generation | artes --ai --aiModel "gemini 2.5 flash" | | --aiKey | API key for the selected AI provider | artes --ai --aiKey "your-api-key" | | --aiURL | Local AI endpoint URL (e.g., Ollama, LM Studio). Overrides --aiModel and --aiKey when set | artes --ai --aiURL "http://localhost:11434/api/chat" | | --aiLanguage | Language for AI-generated reports (default: "English") | artes --ai --aiLanguage "Azerbaijani" | | --maxTokens | Maximum tokens for AI-generated reports (default: 4000) | artes --ai --maxTokens 8000 | | --maxReports | Maximum number of AI reports to generate per test run (default: 10) | artes --ai --maxReports 5 |

** To just run the tests: Globally: artes Locally: npx artes

🎯 Best Practices

• Global Installation:
  For ease of use, it's recommended that Artes be installed globally. You can do this by running the following command:

  npm install -g artes

• Project Creation (Recommended):
  To create a new project with Artes, use the -c flag. This will automatically set up the folder structure and configuration for you. Run the command:

  artes -c

🗂️ Example Project Structure: After running the -c flag to create a new project, the structure will look like this:

/artes (Project Name)
  /tests
    /features
      (Your feature files here)
    /POMs    // Optional
      (POM JSON file here)
    /steps  // For custom steps and hooks
        (Your step definition and hooks JS files here)
  artes.config.js
  /report
    (Generated Allure report HTML here)

If you choose not to use the -c flag, you can still download Artes to your testing project and use the prepared steps by running:

npx artes

You must customize the paths of features, steps, and other configurations by editing the artes.config.js file located inside your project folder (or create it).

For example:

module.exports = {
  paths: ["tests/features/"], // Custom path for feature files
  require: ["tests/steps/*.js"], // Custom path for step definitions files
  pomPath: "tests/POMS/*.js", // Custom path for POM files
};

📝 Writing Feature Files and POM Files

Artes simplifies your test writing with structured feature files and organized Page Object Models (POM). Here’s how you can create them:

1. 📄 Feature File Structure

Feature: Searching on Google 🔍
   Scenario Outline: Search for a term on Google
       Given User is on "https://www.google.com/" page
        When User types "alma" in "google_search_input"
         And User clicks "google_search_button"
         And User waits 10 seconds
        Then "google_text" should have "Alma" text

• Feature: Describes the main feature being tested (e.g., Google search).
• Scenario Outline: Defines a test case with steps.
• Steps: Use Given, When, And, Then keywords to describe actions and expectations.
• Selectors: The element names (e.g., google_search_input, google_search_button) map to the POM file or can be defined directly.

2. 📂 POM File Example

{
  "google_search_input": { "selector": "#APjFqb" },
  "google_search_button": {
    "selector": "input.gNO89b"
  },
  "google_text": {
    "selector": "#rso div h3",
    "waitTime": 5 //seconds
  }
}

• 📑 Using POM File is optional but it is RECOMMENDED
• 🔗 Using Selector in Feature File is possible

  When User types "alma" in "#APjFqb"

• 🐍 It is good to use snake_case for element names
• ⏳ "waitTime" is to define custom wait for elements, but the feature currently under development. "selector" must be used if "waitTime" is used, but when using only selector is not needed mention in "selector"

🔄 Variable Management

Artes provides powerful variable management capabilities that allow you to save, reuse, and share variables across test steps.

Saving Variables from API Responses

You can save variables directly from API responses, even using dot notation to extract nested values:

When User sends GET request to "https://www.test.com" and saves "id" variables
 And User sends GET request to "https://www.test.com/items/{{id}}"
Then User expects that response should have 200 status code

Manual Variable Assignment

Save variables manually using the dedicated step:

And User saves "15" variable as "id"

Variable Randomization

Artes includes built-in randomization for generating dynamic test data:

And User sets random email as "email"
And User sets random 5 words as "description"
And User sets random fullname as "fullName"

Using Variables in Tests

Reference saved variables anywhere in your tests using double curly braces:

When User types "{{email}}" in "email_input"

📚 For detailed information and complete step definitions, visit the Variable Management Documentation.

🛠️ Customization

✍️ Writing Custom Step Definitions

Artes allows you to extend its functionality by writing custom step definitions. Here's how you can do it:

Import Required APIs

const {
  expect,
  Given,
  When,
  Then,
  element,
  context,
  keyboard,
  mouse,
  frame,
  assert,
  elementInteractions,
} = require("artes"); // Common JS
import { expect, Given, When, Then, element, context } from "artes"; // ES Modules (Do not RECOMMENDED)

• Given, When, Then: These define your steps in Cucumber syntax. Example:

  Given("User is on the login page", async () => {
    await context.page.navigateTo("https://example.com/login");
  });

• page: Provides higher-level page actions such as navigation and waiting(Same as PlayWright). Examples:

• Navigate to a URL:

  await context.page.navigate("https://example.com");

• Wait for a selector:

  await context.page.waitForSelector("#loadingSpinner");

• request: Use for sending HTTP requests. (Note: This feature is currently under development.)

• element: Use for interacting with elements on the web page. Examples:

• Clicking a button:

  await element("#submitButton").click();

• Filling an input:

  await element("#username").fill("testUser");

• expect: Use for assertions in your steps. For example:

  expect(actualValue).toBe(expectedValue);
  expect(element("Page_Title")).toHaveText(expectedValue);

📋 Simplified Functions

If you don't want to deal with Playwright methods directly, you can simply use the following predefined actions methods by import them:

const { mouse, keyboard, frame, elementInteractions, page } = require("artes");

• Mouse Actions:
  mouse.click(element)

• Keyboard Actions:
  keyboard.press(key)

• Element Interactions:
  elementInteractions.isChecked()

• Assertions:
  assert.shouldBeTruthy(element)

• Frame Actions:
  frame.first()

• API Actions:
  api.post(url, payload, requestDataType)

For a detailed explanation of each function, please refer to the function definitions.

Example of Custom Step Definition

const { Given, When, Then, expect, element, page } = require("artes");

Given("User is on the home page", async () => {
  await page.navigate("https://example.com");
});

When("User clicks the login button", async () => {
  await element("#loginButton").click();
});

Then("User should see the login form", async () => {
  expect(element("#loginForm")).toBeVisible(true);
});

🪝 Hooks (Lifecycle Hooks)

Artes supports hooks that allow you to execute custom logic before and after tests, scenarios, and steps.

Hooks are user-defined.

📁 Hooks File Location

Create the following file inside your project (optional):

tests/steps/hooks.js

✍️ Writing Hooks

You can define only the hooks you need in hooks.js under the steps folder.
Undefined hooks are automatically skipped.

// tests/steps/hooks.js

export function BeforeStep() {
  // hook for before each step
}

export function Before() {
  // hook for before each test
}

export function BeforeAll() {
  // hook for before all tests
}

export function AfterStep() {
  // hook for after each step
}

export function After() {
  // hook for after each test
}

export function AfterAll() {
  // hook for after all tests
}

🔁 Supported Hook Types

| Hook Name | Execution Time | | ------------ | ----------------------------- | | BeforeAll | Once before all scenarios | | Before | Before each scenario | | BeforeStep | Before each step | | AfterStep | After each step | | After | After each scenario | | AfterAll | Once after all scenarios |

▶️ Execution Order Example

For a scenario with steps:

BeforeAll
Before
BeforeStep
(step executes)
AfterStep
After
AfterAll

⚙️ Configuration

You can configure Artes by editing the artes.config.js file. Below are the default configuration options with explanations:

| Option | Default Value | Description | | ----------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------- | | headless | false | Run browser in headless mode. | | env | "" | Environment name for tests. | | variables | {} | Variables for tests. | | baseURL | "" | Base URL for API or web tests. | | timeout | 30 | Default timeout in seconds. | | slowMo | 0 | Default slow motion in seconds. | | paths | [moduleConfig.featuresPath] | Paths to feature files. | | require | [moduleConfig.stepsPath, "src/stepDefinitions/*.js", "src/hooks/hooks.js"] | Support code paths (CommonJS). | | pomPath | moduleConfig.pomPath | Path to Page Object Models. | | import | [] | Support code paths. | | testPercentage | 0 | Define test coverage percentage | | report | false | Generate report | | zip | false | Generate zip of report | | reportSuccess | false | Add screenshots and video records also for success test cases | | trace | false | Enable tracing | | reportWithTrace | false | Include trace in report | | logo | "" (uses default Artes logo) | Custom logo for the report sidebar. Accepts an absolute path, a relative path, or a direct image URL | | brandName| "" | Brand name displayed next to the logo in the report sidebar | | reportName| "" | Report name displayed on the summary widget | | format | ["rerun:@rerun.txt", "allure-cucumberjs/reporter"] | Formatter names/paths. | | formatOptions | { "resultsDir": "allure-result" } | Formatter options. | | parallel | 1 | Number of parallel workers. | | browser | "chrome" | Browser to use: "chrome", "firefox", "webkit". | | offline | false | Run browser in offline mode. | | device | "" | Emulate specific device (e.g., "iPhone 13"). | | width | 1280 | Browser width in pixels. | | height | 720 | Browser height in pixels. | | maximizeScreen | true | Maximize browser window on start. | | dryRun | false | Prepare test run without execution. | | failFast | false | Stop on first test failure. | | forceExit | false | Force process.exit() after tests | | strict | true | Fail on pending steps. | | backtrace | false | Show full backtrace for errors. | | tags | "" | Tag expression to filter scenarios. | | name | [] | Run scenarios matching regex. | | order | "defined" | Run order (defined/random). | | language | "en" | Default feature file language. | | loader | [] | Module loader specifications. | | requireModule | [] | Transpilation module names. | | retry | 0 | Retry attempts for failing tests. | | retryTagFilter | "" | Tag expression for retries. | | publish | false | Publish to cucumber.io. | | worldParameters | {} | Custom world parameters. |

🤖 AI Configuration

| Option | Default Value | Description | | -------------------- | ----------------- | ---------------------------------------------------------------------------------------------------- | | ai.ai | false | Enable AI-generated bug reports and test summaries. | | ai.model | "gpt-4o" | AI model to use for report generation (e.g. "gemini 2.5 flash", "claude sonnet") | | ai.key | "" | API key for the selected AI provider. | | ai.url | "" | Local AI endpoint URL (e.g. Ollama, LM Studio). Overrides model and key when set. | | ai.language | "English" | Language for AI-generated reports (e.g. "Azerbaijani", "German"). | | ai.maxTokens | 4000 | Maximum tokens for AI response output | | ai.maxReports | 10 | Maximum number of AI reports to generate per test run. |

🌍 Environment Configuration

| Option | Default Value | Description | | ---------- | ----------------- | ------------------------------------------------------------------------------------------------------------ | | env | "" | Environment configuration. Should match the name with the baseURL object, like "dev" | | baseURL | "" | Base URL for API requests. Can be object {"dev":"dev-api.com", "pre":"pre-api.com"}, or string "dev-api.com" |

Environment Variables Configuration

Artes supports environment-specific configurations through environment variables. This feature allows to manage different settings for environments.

Setting Up Environment Variables

1. Configure Environment in artes.config.js:

   module.exports = {
     baseURL: {
       dev: "https://dev.alma.az",
       pre: "https://pre.alma.az",
       prod: "https://api.alma.az",
     },
     env: "dev", // Specify which environment to use
   };

   Alternative single URL configuration:

   module.exports = {
     baseURL: "https://api.alma.az", // Direct string URL
   };

2. Create Environment Variable Files: Create JSON files under src/tests/environment_variables/ folder with names matching your environment:

   dev.env.json:

   {
     "api_key": "dev-api-key-12345",
     "auth_token": "dev-auth-token",
     "database_url": "dev-db.example.com",
     "timeout": 5000,
     "headers": {
       "Authorization": "Bearer dev-token",
       "Content-Type": "application/json"
     }
   }

How It Works

1. Environment Detection: When Artes runs, it reads the env value from artes.config.js
2. Base URL Resolution: If baseURL is an object, it uses the environment key to find the corresponding URL. If baseURL is a string, it uses it directly
3. Variable Loading: Artes looks for a JSON file matching the environment name in src/tests/environment_variables/
4. Runtime Access: All variables from the environment file become available during test execution

Important Notes

• ⚠️ Base URLs must be defined in artes.config.js - they cannot be set in the environment variable JSON files
• 📁 Environment variable files should be placed in src/tests/environment_variables/
• 🏷️ File names must follow the format {env}.env.json (e.g., dev.env.json for env: "dev")
• 🔄 Variables are loaded into variable storage and can be accessed during test runs
• 🌐 Use environment variables for headers, API keys, timeouts, and other environment-specific configurations

Browser Configuration

| Option | Default Value | Description | | ------------- | ------------------------------ | ------------------------------------------------------ | | browserType | "chrome" | Browser type ("chrome", "firefox", or "webkit"). | | viewport | { width: 1280, height: 720 } | Browser viewport size. | | headless | true | Run browser in headless mode (true or false). | | offline | false | Run browser in offline mode (true or false). |

Device Configuration

| Option | Default Value | Description | | ------------- | ------------------------------ | ------------------------------------------------------ | | device | "" | Device List |

📊 Reporting

Artes can generate Allure reports. After running tests with the -r flag, the reports will be stored in the report folder in HTML format. You can view them in your browser after the tests complete.

Integration with Artes Reporting System

Artes has a built-in integration with the Artes Reporting System. By configuring the options below, you can automatically upload your test reports and keep your pipeline stages clean and organized.

| Option | Default Value | Description | | ---------------- | ----------------------------- | ---------------------------------------------------------------------- | | uploadReport | false | Automatically upload the report to Artes Reporting System after tests. | | reporterURL | "" | URL of the Artes Reporting System instance to upload the report to. | | projectName | "Artes Report" | Name of the project in the Artes Reporting System. | | projectType | "Artes" | Type/category of the project (e.g., UI, API). | | reportName | "Artes Report" | Display name of the report in the Artes Reporting System. | | reportPath | "./report.zip" | Path to the report zip file to be uploaded. |

🐳 Docker Image for CI/CD

A Docker image vahidaghayev/artes is available for running Artes in CI/CD pipelines. This image includes:

• Playwright Browsers: Pre-installed to support UI testing.
• Xvfb: Enables running UI tests with video recording in a virtual display.

Recommended Settings for Best Quality

To achieve the best video recording quality, use the following command:

xvfb-run -a --server-args="-screen 0 3840x1180x24" --auto-servernum npx artes --width 1600 --height 900

CI/CD Executor on Report

Artes automatically detects your CI/CD environment and displays executor information — pipeline name, build number, and a direct link to the build — on the generated report.

No configuration needed. Artes reads the standard environment variables that each CI platform sets automatically. Supported platforms include GitHub Actions, Jenkins, GitLab CI, Bitbucket Pipelines, CircleCI, Azure Pipelines, TeamCity, Travis CI, and Bamboo. When running locally, the executor is shown as "Local Run".

For full details, platform-specific examples, and the list of detected environment variables, see the CI/CD Executor Integration guide.

👍 Good To Use

If you don't use the -c or --create option that the package offers, save the file below under the .vscode folder:

• Those configurations will help autocomplete both predefined and custom step definitions in your features file

extensions.json

{
  "recommendations": ["CucumberOpen.cucumber-official"]
}

settings.json

{
  "cucumber.glue": [
    "tests/steps/*.{ts,js}",
    "node_modules/artes/src/tests/stepDefinitions/*.{ts,js}"
  ],
  "cucumber.features": ["tests/features/*.features"],
  "cucumberautocomplete.syncfeatures": true,
  "cucumberautocomplete.strictGherkinCompletion": true
}

🧑‍💻 Have a Good Testing