Monocart Reporter

• A Playwright Test Reporter (Node.js)
  • Shows Test Report in A Tree Grid
  • Console Logs in Order (log/error/warn/debug/info)
  • Custom Annotations with Markdown
  • Timeline Workers Graph
  • Monitor CPU and Memory Usage
  • Export Data (json)
• Preview
• Install
• Playwright Config
• Examples
• Output
• Reporter Options
• View Trace Online
• Custom Fields Report
  • Custom Columns
    • Column Formatter
    • Searchable Fields
  • Custom Fields in Comments
    • Create Diagrams and Visualizations with Mermaid
  • Custom Data Visitor
    • Collect Data from the Title
    • Collect Data from the Annotations
    • Remove Secrets and Sensitive Data
• Style Tags
• Metadata
• Trend Chart
• Attach Lighthouse Audit Report
• Code Coverage Report
  • Global Coverage Report
  • Coverage Options
  • Coverage Examples
• Attach Network Report
• Global State Management
  • Setup Global State
  • Get, Set, and Remove Global Data
  • Send and Receive Messages between Processes
• Merge Shard Reports
• onEnd Hook
  • Send Email
  • Testrail Integration
  • Jira + Zephyr Scale Integration
  • Jira + Xray Integration
  • Slack Integration
  • Discord Integration
  • Teams Integration
  • BrowserStack Integration
  • Dingtalk/Weixin/Feishu Integration
• Contributing
• Changelog

Preview

https://cenfun.github.io/monocart-reporter

Install

npm i -D monocart-reporter

Playwright Config

Note: Most examples use CommonJS by default, please move to ESM according to your needs.

// playwright.config.js
module.exports = {
    reporter: [
        ['list'],
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html'
        }]
    ]
};

Playwright Docs https://playwright.dev/docs/test-reporters

Examples

• tests
• More Examples monocart-reporter-examples

Output

• path-to/your-filename.html
  Single HTML file (data compressed), easy to transfer/deploy or open directly anywhere

Note: Test attachments (screenshots images/videos) are not included but linked with relative path in report. All attachments will be found in playwrightConfig.outputDir

// playwright.config.js
// attachments outputDir and report outputFile used same folder
const date = new Date().toISOString().slice(0, 10); //2022-10-10
const outputDir = `./test-results/${date}`;
module.exports = {
    outputDir: outputDir,
    reporter: [
        ['monocart-reporter', {  
            name: `My Test Report ${date}`,
            outputFile: `${outputDir}/index.html`
        }]
    ]
};
// deploy the folder to your report site and easy checking online
// http://your.report.site/test-results/2022-10-10

• path-to/your-filename.json
  Separated metadata file (Already included in the above HTML and compressed, it can be deleted). Can be used for debugging or custom data collection.

Reporter Options

{
    // the report name
    name: '',

    // the output file path (relative process.cwd)
    outputFile: './test-results/report.html',

    // attachment path handler
    attachmentPath: null,
    // attachmentPath: (currentPath, extras) => `https://another-path/${currentPath}`,

    traceViewerUrl: 'https://trace.playwright.dev/?trace={traceUrl}',

    // logging levels: off, error, info, debug
    logging: 'info',

    // timezone offset in minutes, GMT+0800 = -480
    timezoneOffset: 0,

    // global coverage settings for addCoverageReport API
    coverage: null,
    // coverage: {
    //     entryFilter: (entry) => true,
    //     sourceFilter: (sourcePath) => sourcePath.search(/src\/.+/) !== -1,
    // },

    // trend data handler
    trend: null,
    // trend: () => './test-results/report.json',

    // custom tags style
    tags: null,
    // tags: {
    //     smoke: {
    //         'background': '#6F9913'
    //     },
    //     sanity: {
    //         'background': '#178F43'
    //     }
    // },

    // columns data handler
    columns: null,
    // columns: (defaultColumns) => {},

    // rows data handler (suite, case and step)
    visitor: null,
    // visitor: (data, metadata) => {},

    // enable/disable custom fields in comments. Defaults to true.
    customFieldsInComments: true,

    // onEnd hook
    onEnd: null
    // onEnd: async (reportData, helper) => {}
}

View Trace Online

The Trace Viewer requires that the trace file must be loaded over the http:// or https:// protocols without CORS issue.

• Start a local web server with following CLI:

# serve and open report
npx monocart show-report <path-to-report>

# serve report
npx monocart serve-report <path-to-report>

The server add the http header Access-Control-Allow-Origin: * to allow requesting from any origin, it works with http://localhost:port/ or http://127.0.0.1:port/

• To successfully work with other IP or domain, you can start web server with https:

npx monocart show-report <path-to-report> --ssl <path-to-key,path-to-cert>

For example: npx monocart show-report test-results/index.html --ssl ssl/key.pem,ssl/cert.pem

You can create and install local CA with mkcert

• Using your own trace viewer url with option traceViewerUrl:

// reporter options
{
    name: "My Test Report",
    outputFile: './test-results/report.html',
    // defaults to 'https://trace.playwright.dev/?trace={traceUrl}'
    traceViewerUrl: 'https://your-own-trace-viewer-url/?trace={traceUrl}'
}

Custom Fields Report

You can add custom fields to the report. for example: Owner, JIRA Key etc.

• First, you need to add Custom Columns for the fields.
• Then, collect data for these fields with Custom Fields in Comments or Custom Data Visitor

Custom Columns

The report will be displayed in a Tree Grid. The columns function is used to customize the grid columns. The column properties following:

• id (String) Column id (required)
• name (String) Column name, shows in grid header
• align (String) left (default), center, right
• width (Number) Column width
• minWidth, maxWidth (Number) Default to 81 ~ 300
• styleMap (Object, String) Column style (css)
• formatter (Function) column formatter. Arguments: value, rowItem, columnItem, cellNode
• sortable (Boolean) Column sortable when click column header name
• resizable (Boolean) Column width resizable
• searchable (Boolean) Specifies whether the column is searchable
• markdown (Boolean) Specifies whether the column needs to use markdown conversion
• detailed (Boolean) Specifies whether the column needs to display the layout in detail (horizontal)
• more properties columnProps

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',

            // custom columns
            columns: (defaultColumns) => {

                // insert custom column(s) before a default column
                const index = defaultColumns.findIndex((column) => column.id === 'duration');
                defaultColumns.splice(index, 0, {
                    // define the column in reporter
                    id: 'owner',
                    name: 'Owner',
                    align: 'center',
                    searchable: true,
                    styleMap: {
                        'font-weight': 'normal'
                    }
                }, {
                    // another column for JIRA link
                    id: 'jira',
                    name: 'JIRA Key',
                    width: 100,
                    searchable: true,
                    styleMap: 'font-weight:normal;',
                    formatter: (v, rowItem, columnItem) => {
                        const key = rowItem[columnItem.id];
                        return `<a href="https://your-jira-url/${key}" target="_blank">${v}</a>`;
                    }
                });

            }
        }]
    ]
};

Column Formatter

Note: The formatter function will be serialized into string via JSON, so closures, contexts, etc. will not work!

// playwright.config.js
module.exports = {
     reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            columns: (defaultColumns) => {

                // duration formatter
                const durationColumn = defaultColumns.find((column) => column.id === 'duration');
                durationColumn.formatter = function(value, rowItem, columnItem) {
                    if (typeof value === 'number' && value) {
                        return `<i>${value.toLocaleString()} ms</i>`;
                    }
                    return value;
                };

                // title formatter
                // Note: The title shows the tree style, it is a complicated HTML structure
                // it is recommended to format title base on previous.
                const titleColumn = defaultColumns.find((column) => column.id === 'title');
                titleColumn.formatter = function(value, rowItem, columnItem, cellNode) {
                    const perviousFormatter = this.getFormatter('tree');
                    const v = perviousFormatter(value, rowItem, columnItem, cellNode);
                    if (rowItem.type === 'step') {
                        return `${v}<div style="position:absolute;top:0;right:5px;">✅</div>`;
                    }
                    return v;
                };

            }
        }]
    ]
};

Searchable Fields

// playwright.config.js
module.exports = {
     reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            columns: (defaultColumns) => {
                const locationColumn = defaultColumns.find((column) => column.id === 'location');
                locationColumn.searchable = true;
            }
        }]
    ]
};

Custom Fields in Comments

The code comments are good enough to provide extra information without breaking existing code, and no dependencies, clean, easy to read, etc.

• First, enable option customFieldsInComments to true

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            // enable/disable custom fields in comments. Defaults to true.
            customFieldsInComments: true
        }]
    ]
};

• Then, add comments for the tests

Note: Each comment item must start with @ which is similar to JSDoc.

For example, adding owner and jira to the cases, steps, and suites. or updating the value if the field exists like title

/**
 * for file (comment file in the first line)
 * @owner FO
 */
const { test, expect } = require('@playwright/test');

/**
 * for case
 * @owner Kevin
 * @jira MCR-16888
 */
test('case title', () => { 

});

/**
 * @description multiple lines text description
multiple lines text description
multiple lines text description
 * @jira MCR-16888
*/
test('case description', () => {

});

/**
 * for describe suite
 * @owner Mark
 * @jira MCR-16900
 */
test.describe('suite title', () => {

    test('case title', ({ browserName }, testInfo) => {

        /**
         * rewrite assert step title "expect.toBe" to
         * @title my custom assert step title
         * @annotations important
         */
        expect(testInfo).toBe(test.info());

        // @owner Steve
        await test.step('step title', () => {
        
        });

    });

});

/**
 * rewrite "beforeAll hook" title to
 * @title do something before all
 */
test.beforeAll(() => { 

});

/**
 * rewrite "beforeEach hook" title to
 * @title do something before each
 */
test.beforeEach(() => { 
    
});

Create Diagrams and Visualizations with Mermaid

• Enable Mermaid

// playwright.config.js
module.exports = {
     reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            mermaid: {
                // mermaid script url, using mermaid CDN: https://www.jsdelivr.com/package/npm/mermaid
                scriptSrc: 'https://cdn.jsdelivr.net/npm/mermaid@latest/dist/mermaid.min.js',
                // mermaid config: https://mermaid.js.org/config/schema-docs/config.html
                config: {
                    startOnLoad: false
                }
            }
        }]
    ]
};

• Write Mermaid code in markdown:

/**
 * @description Sequence diagram for Monocart Reporter
```mermaid
flowchart LR

A[Hard] -->|Text| B(Round)
B --> C{Decision}
C -->|One| D[Result 1]
C -->|Two| E[Result 2]
```
*/
test('case description', () => {

});

Custom Data Visitor

The visitor function will be executed for each row item (suite, case and step). Arguments:

• data data item (suite/case/step) for reporter, you can rewrite some of its properties or add more
• metadata original data object from Playwright test, could be one of Suite, TestCase or TestStep

Collect Data from the Title

For example, we want to parse out the jira key from the title:

test('[MCR-123] collect data from the title', () => {

});

You can simply use regular expressions to parse and get jira key:

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            visitor: (data, metadata) => {
                // [MCR-123] collect data from the title
                const matchResult = metadata.title.match(/\[(.+)\]/);
                if (matchResult && matchResult[1]) {
                    data.jira = matchResult[1];
                }
            }
        }]
    ]
};

multiple matches example: collect-data

Collect Data from the Annotations

It should be easier than getting from title. see custom annotations via test.info().annotations

test('collect data from the annotations', () => {
    test.info().annotations.push({
        type: "jira",
        description: "MCR-123"
    })
});

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            visitor: (data, metadata) => {
                // collect data from the annotations
                if (metadata.annotations) {
                    const jiraItem = metadata.annotations.find((item) => item.type === 'jira');
                    if (jiraItem && jiraItem.description) {
                        data.jira = jiraItem.description;
                    }
                }
            }
        }]
    ]
};

Remove Secrets and Sensitive Data

The report may hosted outside of the organization’s internal boundaries, security becomes a big issue. Any secrets or sensitive data, such as usernames, passwords, tokens and API keys, should be handled with extreme care. The following example is removing the password and token from the report data with the string replacement in visitor function.

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            visitor: (data, metadata) => {
                const mySecrets = [process.env.PASSWORD, process.env.TOKEN];
                mySecrets.forEach((secret) => {
                    // remove from title
                    data.title = data.title.replace(secret, '***');
                    // remove from logs
                    if (data.logs) {
                        data.logs = data.logs.map((item) => item.replace(secret, '***'));
                    }
                    // remove from errors
                    if (data.errors) {
                        data.errors = data.errors.map((item) => item.replace(secret, '***'));
                    }
                });
            }
        }]
    ]
};

see example: remove-secrets

Style Tags

• Add tag to test/describe title ( starts with @ )

test('test title @smoke @critical', () => { });
test.describe('describe title @smoke @critical', () => { });

• Custom tag style

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            tags: {
                smoke: {
                    style: {
                        background: '#6F9913'
                    },
                    description: 'This is Smoke Test'
                },
                critical: {
                    background: '#c00'
                }
            }
        }]
    ]
};

Metadata

All metadata will be listed in the report in a key/value format.

• Global level metadata

// playwright.config.js
module.exports = {
    globalSetup: require.resolve('./common/global-setup.js'),
    metadata: {
        product: 'Monocart',
        env: 'STG',
        type: 'Regression',
        executor: 'Mono',
        
        // test home page object model
        url: 'https://www.npmjs.org/package/monocart-reporter'
    },
     reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html'
        }]
    ]
};

• Project level metadata

// playwright.config.js
module.exports = {
    projects: [
        {
            name: 'Desktop Chromium',
            use: {
                browserName: 'chromium'
            },
            metadata: {
                projectData: 'project level metadata',
                owner: 'PO',
                link: 'https://github.com/cenfun/monocart-reporter'
            }
        }
    ]
}

• Collect metadata in global setup or teardown

// ./common/global-setup.js
import { chromium } from '@playwright/test';
export default async (config) => {
    const metadata = config.metadata;
    // collect data and save to global metadata
    const browser = await chromium.launch();
    const chromiumVersion = await browser.version();
    metadata.chromiumVersion = chromiumVersion;
};

• Collect metadata in a test

  Playwright Test runs tests in parallel with isolate test data by default, so we need to utilize global state management to collect metadata in a test.

  • Enable global state, see Setup Global State
  • Update global state in a test

  const { test } = require('@playwright/test');
  const { useState } = require('monocart-reporter');
  
  const state = useState({
      // port: 8130
  });
  
  test('test metadata url', async ({ page }) => {
      const url = await page.evaluate(() => window.location.href);
      await state.set('url', url);
  });

Trend Chart

Note: The trend chart requires historical data generally stored in the server database. There is a serverless solution which is connecting and collecting historical trend data from previous report data before test every time.

• If a report is generated in the same place every time, you can simply connect the data with the report JSON path (the data is not 100% safe if there is any runtime error, the previous output dir will be empty by Playwright but the reporter processing not finish)

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            // connect previous report data for trend chart
            trend: './test-results/report.json'
        }]
    ]
};

• Recommended: resolve the data by yourself (could be requested from the server), required data fields:
  • date (Number) the previous test date in milliseconds
  • duration (Number) the previous test duration
  • summary (Object) the previous test summary
  • trends (Array) historical data list, but except the previous self

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            // connect previous report data for trend chart
            trend: async () => {
                const previousReportData = await readDataFromSomeWhere("path-to/report.json");
                // do some data filtering to previous trends
                previousReportData.trends = previousReportData.trends.filter((item) => {
                    // remove data a week ago
                    return item.date > (Date.now() - 7 * 24 * 60 * 60 * 1000)
                });
                return previousReportData;
            }
        }]
    ]
};

Attach Lighthouse Audit Report

Attach an audit report with API attachAuditReport(runnerResult, testInfo). Arguments:

• runnerResult lighthouse result. see lighthouse
• testInfo see TestInfo

const { test, chromium } = require('@playwright/test');
const { attachAuditReport } = require('monocart-reporter');
const lighthouse = require('lighthouse/core/index.cjs');
test('attach lighthouse audit report', async () => {
    const port = 9222;
    const browser = await chromium.launch({
        args: [`--remote-debugging-port=${port}`]
    });
    const options = {
        // logLevel: 'info',
        // onlyCategories: ['performance', 'best-practices', 'seo'],
        output: 'html',
        port
    };
    const url = 'https://github.com/cenfun/monocart-reporter';
    const runnerResult = await lighthouse(url, options);
    await browser.close();
    await attachAuditReport(runnerResult, test.info());
});

Code Coverage Report

The reporter integrates monocart-coverage-reports for coverage reports, there are two APIs:

• addCoverageReport(data, testInfo) Add coverage to global coverage report from a test. see Global Coverage Report
  • data There are two supported data inputs: Istanbul (Object) or V8 (Array)
  • testInfo see TestInfo
• attachCoverageReport(data, testInfo, options) Attach a coverage report to a test. Arguments:
  • data same as above
  • testInfo same as above
  • options (Object) see Coverage Options

Global Coverage Report

The global coverage report will merge all coverages into one global report after all the tests are finished.

• The global coverage options see Coverage Options

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            // global coverage report options
            coverage: {
                entryFilter: (entry) => true,
                sourceFilter: (sourcePath) => sourcePath.search(/src\/.+/) !== -1,
            }
        }]
    ]
};

• Adding coverage data for each test with automatic fixtures

// fixtures.js for v8 coverage
import { test as testBase, expect } from '@playwright/test';
import { addCoverageReport } from 'monocart-reporter';

const test = testBase.extend({
    autoTestFixture: [async ({ page }, use) => {

        // NOTE: it depends on your project name
        const isChromium = test.info().project.name === 'Desktop Chromium';

        // console.log('autoTestFixture setup...');
        // coverage API is chromium only
        if (isChromium) {
            await Promise.all([
                page.coverage.startJSCoverage({
                    resetOnNavigation: false
                }),
                page.coverage.startCSSCoverage({
                    resetOnNavigation: false
                })
            ]);
        }

        await use('autoTestFixture');

        // console.log('autoTestFixture teardown...');
        if (isChromium) {
            const [jsCoverage, cssCoverage] = await Promise.all([
                page.coverage.stopJSCoverage(),
                page.coverage.stopCSSCoverage()
            ]);
            const coverageList = [... jsCoverage, ... cssCoverage];
            // console.log(coverageList.map((item) => item.url));
            await addCoverageReport(coverageList, test.info());
        }

    }, {
        scope: 'test',
        auto: true
    }]
});
export { test, expect };

• Adding server side coverage on global teardown

For example, a Node.js web server start at the beginning of the test with the env NODE_V8_COVERAGE=dir, the V8 coverage data will be saved to dir with calling API v8.takeCoverage() manually or terminating server gracefully. On global teardown, reading all from dir and adding them to global coverage report. For Node.js, the V8 coverage data requires appending source manually.

// global-teardown.js
import fs from 'fs';
import path from 'path';
import { fileURLToPath } from 'url';
import { addCoverageReport } from 'monocart-reporter';

export default async (config) => {

    const dir = "your-v8-coverage-data-dir";

    const files = fs.readdirSync(dir);
    for (const filename of files) {
        const content = fs.readFileSync(path.resolve(dir, filename)).toString('utf-8');
        const json = JSON.parse(content);
        let coverageList = json.result;
        coverageList = coverageList.filter((entry) => entry.url && entry.url.startsWith('file:'));

        // appending source
        coverageList.forEach((entry) => {
            entry.source = fs.readFileSync(fileURLToPath(entry.url)).toString('utf8');
        });

        // there is no test info on teardown, just mock one with required config
        const mockTestInfo = {
            config
        };
        await addCoverageReport(coverageList, mockTestInfo);
    }
}

see Node.js V8 Coverage Report for Server Side

Coverage Options

• Default options
• Available Reports
• Using entryFilter and sourceFilter to filter the results for V8 report
• Checking thresholds with onEnd Hook
• More Introduction monocart-coverage-reports

Coverage Examples

• For Playwright component testing:
  • playwright-ct-vue
  • playwright-ct-react
  • playwright-ct-svelte
• nextjs-with-playwright
• code-coverage-with-monocart-reporter

Attach Network Report

Attach a network report with API attachNetworkReport(har, testInfo). Arguments:

• har HAR path (String) or HAR file buffer (Buffer). see HAR 1.2 Spec
• testInfo see TestInfo

Generate HAR with recordHar option in browser.newContext() (see example: report-network.spec.js preview report)

const fs = require('fs');
const path = require('path');
const { test } = require('@playwright/test');
const { attachNetworkReport } = require('monocart-reporter');
let context;
test.describe('attach network report 1', () => {

    const harPath = path.resolve('.temp/network-report1.har');
    if (fs.existsSync(harPath)) {
        // remove previous
        fs.rmSync(harPath);
    }

    test('first, open page', async ({ browser }) => {
        context = await browser.newContext({
            recordHar: {
                path: harPath
            }
        });
        const page = await context.newPage();
        await page.goto('https://github.com/cenfun/monocart-reporter');
    });

    test('next, run test cases', async () => {
        
    });

    test('finally, attach HAR', async () => {
        // Close context to ensure HAR is saved to disk.
        await context.close();
        await attachNetworkReport(harPath, test.info());
    });
});

Generate HAR with playwright-har

import { test } from '@playwright/test';
import { attachNetworkReport } from 'monocart-reporter';
import { PlaywrightHar } from 'playwright-har';

const harPath = path.resolve('.temp/network-report2.har');
if (fs.existsSync(harPath)) {
    // remove previous
    fs.rmSync(harPath);
}

test('first, open page', async ({ browser }) => {
    const context = await browser.newContext();
    const page = await context.newPage();
    playwrightHar = new PlaywrightHar(page);
    await playwrightHar.start();
    await page.goto('https://github.com/cenfun/monocart-reporter');
});

test('next, run test cases', async () => {
    
});

test('finally, attach HAR', async () => {
    await playwrightHar.stop(harPath);
    await attachNetworkReport(harPath, test.info());
});

Preview Network HTML Report

Global State Management

When tests are executed in isolation mode, the reporter and each test may run in a different process, they cannot share data with each other. we can start a local WebSocket server to serve the global data, and read/write the global data with useState API from a test.

Setup Global State

module.exports = {
    reporter: [
        ['list'],
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            state: {
                data: {
                    count: 0
                },
                server: {
                    // port: 8130
                },
                onClose: (data, config) => {
                    // save state data to global metadata
                    Object.assign(config.metadata, data);
                }
            }
        }]
    ]
};

Get, Set, and Remove Global Data

const { test } = require('@playwright/test');
const { useState } = require('monocart-reporter');
test('state test', async ({ browserName }) => {
    const state = useState({
        // port: 8130
    });

    const count = await state.get('count');
    console.log('count', count);

    await state.set('count', count + 1);

    await state.set({
        browser: browserName,
        someKey: 'some value'
    });

    const [browser, someKey] = await state.get('browser', 'someKey');
    console.log(browser, someKey);

    await state.remove('someKey');

    const all = await state.get();
    console.log(all);
});

Send and Receive Messages between Processes

• send message and receive response from a test (child process)

const { test } = require('@playwright/test');
const { useState } = require('monocart-reporter');
test('state test send message', async () => {
    const state = useState({
        // port: 8130
    });
    const response = await state.send({
        testId: test.info().testId,
        data: 'my test data'
    });
    console.log('receive response on client', response);
});

• receive message and send response from global state (main process)

module.exports = {
    reporter: [
        ['list'],
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            state: {
                onReceive: function(message) {
                    const test = this.getTest(message.testId);
                    if (test) {
                        // current test
                    }
                    console.log('receive message on server', message);
                    return {
                        data: 'my response data'
                    };
                }
            }
        }]
    ]
};

see example: Allow specified test cases to run in sequence mode with lock/unlock state

Merge Shard Reports

There will be multiple reports to be generated if Playwright test executes in sharding mode. for example:

npx playwright test --shard=1/3
npx playwright test --shard=2/3
npx playwright test --shard=3/3

There are 3 reports will be generated. Using merge(reportDataList, options) API to merge all reports into one.

Note: One more suite level "shard" will be added, its title will be the machine hostname, and the summary will be restated. You may need to transfer the attachments by yourself and update the path of the attachments with attachmentPath API.

import { merge } from 'monocart-reporter';

const reportDataList = [
    // json file path
    'path-to/shard1/index.json',
    'path-to/shard2/index.json',
    // or JSON data
    JSON.parse(fs.readFileSync(path.resolve('path-to/shard3/index.json')))
];

await merge(reportDataList, {
    name: 'My Merged Report',
    outputFile: 'merged-report/index.html',
    attachmentPath: (currentPath, extras) => {
       // return `https://cenfun.github.io/monocart-reporter/${currentPath}`;
    },
    onEnd: async (reportData, helper) => {
        // send email or third party integration
    }
});

see example merge.js

onEnd Hook

The onEnd function will be executed after report generated. Arguments:

• reportData all report data, properties:
  • name (String) report name
  • date (Number) start date in milliseconds
  • dateH (String) human-readable date with Date.toLocaleString()
  • duration (Number) test duration in milliseconds
  • durationH (String) human-readable duration
  • summary (Object) test summary, includes tests, suites, steps, etc.
  • rows and columns (Array) all rows and columns data, both are tree structure, see detail
  • tags (Object) tag collection
  • metadata (Object) metadata collection
  • system (Object) system information
  • trends (Array) historical trend data
  • caseTypes and suiteTypes (Array)
  • cwd, outputDir and outputFile (String)
  • htmlPath, jsonPath and summaryTable (String)
  • ...
• helper APIs:
  • helper.find(callback) Find item like array find function.
  • helper.filter(callback) Filter list like array filter function.
  • helper.forEach(callback) Iterate all rows of data (suites/cases/steps), return break will break the iteration.
  • helper.sendEmail(emailOptions)

// playwright.config.js
module.exports = {
    reporter: [
        ['monocart-reporter', {  
            name: "My Test Report",
            outputFile: './test-results/report.html',
            // async hook after report data generated
            onEnd: async (reportData, helper) => {
                // console.log(reportData.summary);

                // find a test by title
                const myCase = helper.find((item, parent) => item.type === 'case' && item.title.includes('inline tag'));
                console.log(myCase && myCase.title);

                // find a suite by title
                const mySuite = helper.find((item, parent) => item.type === 'suite' && item.title.includes('new syntax'));
                console.log(mySuite && mySuite.title);

                // filter failed cases
                const failedCases = helper.filter((item, parent) => item.type === 'case' && item.caseType === 'failed');
                console.log(failedCases.map((it) => it.title));

                // Iterate all items
                helper.forEach((item, parent) => {
                    // do something
                });


            }
        }]
    ]
};

Send Email

• Simply send email with nodemailer
• Example: send-email

Testrail Integration

• Send test results to your Testrail
• Example: testrail

Jira + Zephyr Scale Integration

• Create test cycle and executions with zephyr-scale-api
• Example: zephyr-scale

Jira + Xray Integration

• Import test execution results with Xray REST API
• Update Jira issue status with Jira Transition API
• Example: xray

Slack Integration

• Simply send message with @slack/webhook
• Example: slack-webhook
• Post chat message and upload image with @slack/web-api
• Example: slack-web-api

Discord Integration

• Using Discord webhooks to post messages to channels.
• Example: discord-webhook

Teams Integration

• Please create an Incoming Webhooks for the channel first.
• Example: teams-webhook

BrowserStack Integration

• Example: browserstack

Dingtalk/Weixin/Feishu Integration

• dingtalk-webhook
• weixin-webhook
• feishu-webhook

Contributing

# Node.js 20+
npm install starfall-cli -g
npm install

npm run build
npm run test

npm run dev

Dependencies

• UI Framework Vue 3
• Lightweight UI Components vine-ui
• High Performance Grid turbogrid
• String compress/decompress lz-utils