feature-cup

Create feature coverage reports for your test automation system. We are aware that reporting is a key component for test automation suites, however one important question that is usually not answered is "how much of your system is covered by these tests?" or "what features are not covered and we should verify manually?".

feature-cup allows us to specify the functional structure of the system-under-test and then match the test automation code to the functional structure. The result is a feature coverage report that shows how much of our system is covered by those tests.

Note that feature-cup does not require tests to run, as it does a static analysis of the test code.

Installation

npm install feature-cup --save-dev

Usage

Find the help by running:

feature-cup --help

Here is a basic usage of the reporter:

feature-cup --features ./demo/features.json --tests "./demo/specs/**/*.spec.js" --output console

features.json

The features.json file is a json file that contains the system functional structure. It is used to create the feature coverage report. The file is a json array of module objects, such as:

{
  "system": {
    "name": "My System",
    "modules": [
      {
        "name": "Module A",
        "features": [
          {
            "name": "Feature 1",
            "description": "This is feature 1 of module A"
          }
        ]
      }
    ]
  }
}

Tests

The tests are the files that contain the test automation code. They may be written for mocha, jasmine or jest test runners. This reporter will parse all the "describe" blocks in the test files and match them to the features in the features.json file. The reporter will then output the feature coverage report.

The tests are specified using the --tests option. The tests are specified using the glob pattern. For example, to specify all the files in the "specs" folder, use:

--tests "./specs/**/*.spec.js"

In the demo folder, we have the following test files:

./demo
├──demo.js // usage through a NodeJS file
├──features.json // the functional structure of the system
├──reports
│  ├──feature-coverage-report.html // sample html report
└──specs // the test files
   ├──module_a.spec.js
   ├──module_b // feature-cup will find all test files recursively
   │  ├──module_b.spec.js
   │  ├──module_b_feature_4.spec.js
   │  └──module_b_feature_6.spec.js
   ├──module_c.spec.js
   └──module_f
      └──module_f.spec.js // all "describe" blocks in files will be matched to the features in the features.json file

Output

The output is specified using the --output option. The output can be "console" or "html".

If the output is "html", then the --report option may be specified. The report option is the path where report will be generated. If not specified, report will be saved in "reports" folder. For example:

--output console
--output html --report feature-coverage

Examples

Find the source of examples in the repo: https://github.com/oscarcenteno/feature-cup-example

# Sample output report to console
feature-cup --features ./demo/features.json --tests "./demo/specs/**/*.spec.js" --output console

# Sample output report to html file
feature-cup --features ./demo/features.json --tests "./demo/specs/**/*.spec.js" --output html --report feature-coverage

will output as an html report at demo/reports/feature-coverage.html:

)

Development

Install dependencies

npm install

Run tests

Business and presentation logic are covered by unit tests.

npm test

Static code checking

npm run typecheck # check typescript types
npm run lint # check code style

Get code coverage report

npm run coverage

Coverage information is reported to coveralls.io and can be found at https://coveralls.io/github/oscarcenteno/feature-cup.

# Command used by github actions to report coverage on each commit to main branch, see .github/workflows/nodejs.yml
npm run ci:coverage

End to End testing

Here, make sure everything works together.

npm run demo:console # check no errors are shown in the console
npm run demo:html #check the report in the "reports" folder

Contributors

• [Oscar Centeno] (https://github.com/oscarcenteno)

License

MIT

References

• Mocha
• Jasmine
• Jest
• Istanbul
• Chai
• ESLint