Playwright (TS binding) + Cucumber (BDD)

Cucumber is a popular behavior-driven development (BDD) tool that allows developers and stakeholders to collaborate on defining and testing application requirements in a human-readable format. TypeScript is a powerful superset of JavaScript that adds optional static typing, making it easier to catch errors before runtime. By combining these two tools, we can create more reliable and maintainable tests.

Features

1. Awesome report with screenshots, videos & logs
2. Execute tests on multiple environments
3. Parallel execution
4. Rerun only failed features
5. Retry failed tests on CI
6. Github Actions integrated with downloadable report
7. Page object model

Sample report

Project structure

• app -> Contains all the Cucumber features & Step Definitions and Playwriht Tests
• dist -> Contains all framework code output files
• tm-playwright-framework/dist -> Contains all framework code source files
• cucumber-results -> Contains all the Cucumber reports related file
• playwright-test-results -> Contains all the Playwright reports related file

Reports

1. Mutilple Cucumber Report
2. Default Cucumber report
3. Logs
4. Screenshots of failure
5. Test videos of failure
6. Trace of failure

Get Started

Setup:

1. Clone or download the project
2. Extract and open in the VS-Code
3. npm i to install the dependencies
4. npx playwright install to install the browsers
5. npm run ccs_cucumber_test to execute the Cucumber tests or npm run ccs_playwright_test to execute te Playwriht tests
6. To run a particular test pass FEATURES as a command line arument with required feature
7. Use TAGS command line parameter to run a specific or collection of specs for Cucumber Tests

Browser launch environment variables

• MAXIMIZED_BROWSER=true — launches supported browsers in maximized mode.
• DISABLE_WEB_AUTH=true — adds Chrome/Chromium/Firefox flags to disable WebAuthentication during the run.
• CHROME_ARGUMENTS=[] — additional Chrome/Chromium launch arguments. e.g. CHROME_ARGUMENTS=["--disable-features=WebAuthn","--disable-webauthn"]

These values are configured in app/env/.env.prod and are used by tm-playwright-framework/dist/browser/browserManager.ts.

Folder structure - May not be up to Date until folder structure is finalized.

0. .vscode -> Settings for feature and step definition mapping
1. app -> Folder for application related code and files
2. app\env -> Multiple environments are handled
3. app\pom\objects -> Page Object Model classes to have Object Locators
4. app\pom\scripts -> Page Object Model classes to have custom methods related to the page
5. pom.ts -> Global Objects and Scripts
6. app\test-data -> Folder to have test data and test upload files
7. app\test\features\<folder>\ -> write your features here (format: featureName.feature. You can have folders based on need)
8. app\test\steps\<folder>\ -> Step definitions can be under the Feature folder here (format: featureName.step.ts)
9. app\test\playwright\ -> Playwright tests can be under this folder
10. dist -> Compiled application code files generated here. No need to update/change anything here
11. node_modules -> Dependency libraries are installed here
12. tm-playwright-framework -> Framework code files. Development or changes should be made here
13. cucumber-results -> Folder to have execution reports and logs of Cucumber Tests
14. cucumber-results\default-report -> Cucumber default report
15. cucumber-results\myltiple-cuucmber-report -> Multiple Cucumber report
16. cucumber-results\myltiple-cuucmber-report\assets -> CSS and font files for Multiple Cucumber report
17. playwright-test-results\ -> Folder to have execution reports of Playwright Tests
18. package.json -> Contains all the dependencies
19. tsconfig.json -> TypeScript configurations

Core Framework

1. actions -> Keyboard, Mouse and other user Action's on elements
   1. custom -> Custom action like Alert handling and etc.
   2. dropdown -> Dropdown actions
   3. element -> Methods depends on Elements e.g. Wait till disappearance of some element
   4. keyboard -> Keyboard actions
   5. mouse -> Mouse actions
   6. checkbox -> Check box actions
   7. dropdown -> Dropdown actions
   8. radiobutton -> Option button actions
   9. fileupload -> File Upload actions
2. assertion -> Collection of Assertion
   1. assertion -> Text and various Assertions
   2. email -> Read email and assert email content
3. browser -> Manage browser invoke and teardown
   1. browsermanager -> Manage browser items
4. config -> Configuration files
   1. cucumber.js -> Cucumber profile configuration for test runner
5. helper -> Cross ENV and DotEnv configurations
   1. env.ts -> Environment binding for Env variables
   2. types -> Type definition for Env variables
6. hooks -> Cucumber hooks
   1. `hooks' -> Cucumber hooks implementation
   2. `pageFixture' -> Fixture and Page setup
7. report -> Report and logger configurations
   1. 'custom_logger` -> Custom method for log generation
   2. init.ts -> Report initialization
   3. logger.ts -> Winston logger
   4. report.ts -> Cucumber Report Configuration
8. runner -> Runner configurations
   1. cucumber-runner.ts -> Receives command line arguments and execute Cucumber Tests as defined
   2. plyawright-runner.ts -> Receives command line arguments and execute Playwright Tests as defined
9. global.ts -> Global access variables and class members and scripts10.

Running Cucumber Test through Terminal (Command Prompt)

Syntax: npm run ccs_cucumber_test -- --ENV={Env} [--FEATURES={Feature}] [--TAGS={tags}] [--BROWSER={Chrome|Firefox|Webkit}] [--HEADLESS={true|false}] [--CLIENT={client}] [--REPORT_PATH={Report_Path}] [PARALLEL_WORKER={PARALLEL_WORKER_Count}] - ENV is mandatory (required) argument. Value can be any one of the following: dev, test, qa, stage, ct, uat, prod - If no FEATURES are specified then it will run all features - If no TAGS are specified then it will run all scenarios of applicable features irrespective of tag - If no BROWSER is specified then default would be Chrome. Value can be only Chrome or Firefox or Webkit - If HEADLESS parameter is not specified then default would be Headless as false - If CLIENT parameter is not specified then it will be ignored - If REPORT_PATH parameter is not specified then report will be generated at folder test-results/default-report. Otherwise Cucumber default report HTML and JSON files will b generated at the specified location - If PARALLEL_WORKER parameter is not specified then default would be 1. If specified then given number of parallel execution will occur

• The following will not work as ENV variable is mandatory npm run ccs_cucumber_test
• To execute all of the features with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD
• To execute all of the features with PROD environment variables with Chrome for a specific client npm run ccs_cucumber_test -- --ENV=PROD --CLIENT=TransAmerica
• To execute all of the features under loginPage folder with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD --FEATURES=app/test/features/loginPage
• To execute all of the features under loginPage and booksPage folder with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD --FEATURES=app/test/features/loginPage,app/test/features/booksPage
• To execute all of the features under loginPage folder which are tagged as @auth with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD --FEATURES=app/test/features/loginPage --TAGS="@auth"
• To execute all of the features which are tagged as @auth with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD --TAGS="@auth"
• To execute all of the features which are tagged as @auth or @add with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD --TAGS="@auth or @add"
• To execute all of the features which are not tagged as @auth with PROD environment variables with Chrome npm run ccs_cucumber_test -- --ENV=PROD --FEATURES=app/test/features/loginPage --TAGS="not @auth"
• To execute all of the features with PROD environment variables with Firefox npm run ccs_cucumber_test -- --ENV=PROD --BROWSER=Firefox
• To execute all of the features with PROD environment variables with Webkit npm run ccs_cucumber_test -- --ENV=PROD --BROWSER=Webkit
• To execute all of the features with PROD environment variables with browser as Headless npm run ccs_cucumber_test -- --ENV=PROD --BROWSER=Webkit --HEADLESS=true
• To execute all of the features with PROD environment variables with browser as Headless and to generate the Report at downloads folder npm run ccs_cucumber_test -- --ENV=PROD --BROWSER=Webkit --HEADLESS=true --REPORT_PATH=C:/users/Test/Downloads
• To execute all of the features with PROD environment variables with browser as Headless in Parallel mode npm run ccs_cucumber_test -- --ENV=PROD --BROWSER=Webkit --HEADLESS=true --PARALLEL_WORKER=2

TAGS can be passed in different ways

| Input Tag | Converted as | | ------------------------- | --------------------------- | | @HORIZON or @HORIZONSTG | @HORIZON or @HORIZONSTG | | HORIZON or HORIZONSTG | @HORIZON or @HORIZONSTG | | HORIZON and HORIZONSTG | @HORIZON and @HORIZONSTG | | HORIZON, HORIZONSTG | @HORIZON or @HORIZONSTG | | ORACLE and DB | @ORACLE and @DB | | ANDROID | @ANDROID | | not HORIZON | not @HORIZON |

Running Playwright Test through Terminal (Command Prompt)

Syntax: npm run ccs_playwrigt_test -- --ENV={Env} [--TEST_DIR={Test_Dir}] [--TAGS={tags}] [--BROWSER={Chrome|Firefox|Webkit}] [--HEADLESS={true|false}] [--CLIENT={client}] [--REPORT_PATH={Report_Path}] [PARALLEL_WORKER={PARALLEL_WORKER_Count}] [RETRY_COUNT={retry_count}] [REPEAT_EACH={repeat_each}] [REPORTER={reporter}] [TIME_OUT={timetout}] - ENV is mandatory (required) argument. Value can be any one of the following: dev, test, qa, stage, ct, uat, prod - If no TEST_DIR are specified then it will run all playwright tests - If no TAGS are specified then it will run all scenarios of applicable features irrespective of tag - If no BROWSER is specified then default would be Chrome. Value can be only Chrome or Firefox or Webkit - If HEADLESS parameter is not specified then default would be Headless as false - If CLIENT parameter is not specified then it will be ignored - If REPORT_PATH parameter is not specified then report will be generated at folder playwright-test-results. Otherwise Playwright HTML report and JSON file will be generated at the specified location - If PARALLEL_WORKER parameter is not specified then default would be 1. If specified then given number of parallel execution will occur - If REPEAT_EACH parameter is not specified then default would be 1. If specified then every tests will run given no. of times - If RETRY_COUNT parameter is not specified then default would be 1. If specified then given number of times failed tests will be re-executed - If TIME_OUT parameter is not specified then default would be 1. If specified then given timetout value is utilized for each tests

• The following will not work as ENV variable is mandatory npm run ccs_playwrigt_test
• To execute all of the features with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD
• To execute all of the Tests with PROD environment variables with Chrome for a specific client npm run ccs_playwrigt_test -- --ENV=PROD --CLIENT=TransAmerica
• To execute all of the Tests under loginPage folder with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD --TEST_DIR=app/test/playwright/loginPage
• To execute all of the Tests under loginPage and booksPage folder with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD --TEST_DIR=app/test/playwright/loginPage,app/test/playwright/booksPage
• To execute all of the Tests under loginPage folder which are tagged as @auth with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD --TEST_DIR=app/test/playwright/loginPage --TAGS="@auth"
• To execute all of the Tests which are tagged as @auth with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD --TAGS="@auth"
• To execute all of the Tests which are tagged as @auth or @add with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD --TAGS="@auth or @add"
• To execute all of the Tests which are not tagged as @auth with PROD environment variables with Chrome npm run ccs_playwrigt_test -- --ENV=PROD --TES_DIR=app/test/playwright/loginPage --TAGS="not @auth"
• To execute all of the features with PROD environment variables with Firefox npm run ccs_playwrigt_test -- --ENV=PROD --BROWSER=Firefox
• To execute all of the features with PROD environment variables with Webkit npm run ccs_playwrigt_test -- --ENV=PROD --BROWSER=Webkit
• To execute all of the features with PROD environment variables with browser as Headless npm run ccs_playwrigt_test -- --ENV=PROD --BROWSER=Webkit --HEADLESS=true
• To execute all of the features with PROD environment variables with browser as Headless and to generate the Report at downloads folder npm run ccs_playwrigt_test -- --ENV=PROD --BROWSER=Webkit --HEADLESS=true --REPORT_PATH=C:/users/Test/Downloads
• To execute all of the features with PROD environment variables with browser as Headless in Parallel mode npm run ccs_playwrigt_test -- --ENV=PROD --BROWSER=Webkit --HEADLESS=true --PARALLEL_WORKER=2

Tutorials

1. To know more about Framework - [tm-playwright-framework] (https://toppanmerrill.atlassian.net/wiki/spaces/MCSAUT/pages/4084924498/PLAYWRIGHT+FRAMEWORK+ARCHITECTURE)
2. Learn Playwright - Playwright - TS
3. BDD in detail - TS binding