Litmus-Agent

A CLI tool to triage Playwright test failures using playwright JSON report and traces.

Authors

The package is created and published by Litmus Check AI To use this package, the LITMUS_API_KEY can be generated after signing up, or by reaching out to [email protected]

Installation

Global Installation (for CLI usage)

npm install -g litmus-agent

Or install locally in your project:

npm install --save-dev litmus-agent

Setup for playwright

The playwright configuration needs to be setup to record JSON reports and traces for failures. Nodejs is required to run this package.

• Node.js (v14+)
• Playwright JSON report format (generated with ['json', { outputFile: 'report.json' }] reporter)
• Trace files must be generated (configure with trace: 'retain-on-failure' in playwright.config)

Sample config

export default defineConfig({
	testDir: './tests',
	retries: 0,
	reporter: [
		['json', { outputFile: './reports/report.json' }]
	],
	use: {
		trace: 'retain-on-failure',
		screenshot: 'only-on-failure',
	},
	outputDir: './traces',
	projects: [
		{
			name: 'chromium',
			use: { ...devices['Desktop Chrome'] }
		}
	]
});

API Key setup

Environment variables can be set in a .env file in the root directory or as system environment variables.

• LITMUS_API_KEY - API key for Litmus API (required) can be generated on the web app

Usage

After installation, the lc command will be available globally (or via npx if installed locally).

Basic Usage

lc triage <path-to-report.json>

With Options

# Save API response to file with pretty printing
lc triage ./reports/report.json --output response.json --pretty

# Output API response to stdout (default)
lc triage ./reports/report.json

Note: The CLI always sends data to the Litmus API and outputs the API response. If the API call fails, it outputs {error: 'Failed to complete triaging'}.

Options

• <report-path> - Path to Playwright JSON report file (required)
• -o, --output <file> - Output file path for API response (default: stdout)
• -p, --pretty - Pretty-print JSON output
• -c, --error-context <number> - Number of context lines for error snippets (default: 10)
• -V, --version - Output version number
• -h, --help - Display help

Example .env file

Create a .env file in the root directory:

LITMUS_API_KEY=your-api-key-here

Output

The CLI always sends the triage data to the Litmus API and outputs:

• On success (HTTP 200): The API response JSON
• On failure: {error: 'Failed to complete triaging'}

Dependencies

• yauzl - Unzip trace files
• commander - CLI argument parsing
• @babel/parser - Parse TypeScript/JavaScript to extract test functions
• @babel/traverse - Traverse AST to find test blocks
• dotenv - Load environment variables from .env file

License

Attribution-NonCommercial-ShareAlike 4.0 International