protractor-retry-with-failed-spec-deletion

The main benefit of the package has the option to delete failed specs before the subsequent re-run, so in the result, you will be able to create a full report of your run, and you will see there are not just failed tests but all the tests you ran.

You just need to pass additional parameters to the function afterLaunch. It was tested only with Mocha.

• A solution to address the flakiness of your Protractor FE automation test suites.
• This module used protractor features to automatically re-run failed tests with a specific configurable number of attempts.
• This module is added in our CI/CD pipelines where we have a zero failure policy in order to bless an environment.
• Mocha & Jasmine are supported.
• NEW : WINDOWS Support UPDATE : Windows as an env to launch with version 2.0.1

This module fully relies on protractor available callbacks and internal functionalities.

It is build from within protractor itself and not relying on any external dependency. All the changes that you need to add in order to integrate this module is located only in one file, the protractor configuration file, ie to summarize :

• You don’t need to update a single line of your tests spec.js files.
• Only few lines to add in your protractor conf file needed.

The module will create an XML file which contains the failed spec(s) filename(s). and will re-run only them, till either we don't have anymore failures or we reached the retry attempt maximum.

The process of retrying is not happening on the fly of a test failure but only after the whole testsuite is run. The failed tests are stored and only those ones are going to be rerun afterwards by creating on the fly a new "failed only" files testsuite.

Steps to Integrate

Install

npm i protractor-retry-with-failed-spec-deletion

Step 1: Require the Package ( Your Protractor Config )

var retry = require('protractor-retry-with-failed-spec-deletion').retry;

Step 2: onPrepare ( Your Protractor Config )

onPrepare: function() {
  retry.onPrepare();
}

Step 3: onCleanUp ( Your Protractor Config )

onCleanUp = function(results) {
    retry.onCleanUp(results);
};

It is Mandatory to provide the results to the retry.onCleanUp function

Step 4: afterLaunch ( Your Protractor Config )

afterLaunch = function() {
   return retry.afterLaunch(NUMBER_OF_RETRIES);
}

If you want to have a full report after your test run, you need to pass two additional parameters to afterLaunch to delete failed specs before re-run, so you have no doubling of your failed specs:

afterLaunch = function() {
   return retry.afterLaunch(NUMBER_OF_RETRIES, true, PATH_TO_MOCHA_JSON_FOLDER_WITH_SPEC_REPORTS);
}

It is Mandatory to use return here

Full Protractor Config snippet with 2 retries and failed spec deletion before subsequent re-run:

exports.config = {
    // rest of your config
    onCleanUp: (results) => {
        retry.onCleanUp(results);
    },
    onPrepare: () => {
        retry.onPrepare();
    },
    afterLaunch: () => {
        return retry.afterLaunch(2, true, PATH_TO_MOCHA_JSON_FOLDER_WITH_SPEC_REPORTS);
    }
};

Full Protractor Config snippet with 2 retries:

exports.config = {
    // rest of your config
    onCleanUp: (results) => {
        retry.onCleanUp(results);
    },
    onPrepare: () => {
        retry.onPrepare();
    },
    afterLaunch: () => {
        return retry.afterLaunch(2);
    }
};

Examples

• Checkout this Mocha (Firefox capability) Example protractor.mocha.conf.js
• Checkout this Jasmine (Chrome cap.) Example protractor.jasmine.conf.js
• Checkout this TestSuite (IE11 cap.) Example with DEBUG mode activated protractor.suite.conf.js

Known Caveat

• If you are NOT Running in Parallel mode, the package will retry the whole testsuite if any failure.
• Windows as an environment to launch & use this package is unfortunately not yet supported.
• WINDOWS SUPPORT UPDATE Supported from version 2.0.1