CI TestGen

This project is designed to automate the generation, showing, and applying of tests for your GitHub and GitLab projects.

Prerequisites

• Node.js (version 20 or higher)
• GitHub or GitLab account with appropriate permissions

How to Use

This tool is available via npx but is meant to be used in the CI/CD pipeline. Once you have finished setting up the tool with GitHub / GitLab, the workflow is as follows:

1. Create or Update a Pull/Merge Request: When a developer creates or updates a pull/merge request, the generate-tests command will automatically run in the CI/CD pipeline. This will generate tests based on the changes and post them as comments in the pull/merge request.

2. Review Generated Tests: The developer will see the generated tests as comments in the pull/merge request. They can review these tests and decide which ones to approve.

3. Approve Tests: To approve a test, the developer should react with a 👍 emoji to the comment containing the test they want to approve.

4. Trigger Test Application: After approving the desired tests, the developer should type "testgen" into a new comment in the pull/merge request. This will trigger the apply-tests command in the CI/CD pipeline.

5. Apply Approved Tests: The apply-tests command will apply the approved tests and push the changes to the branch associated with the pull/merge request.

Setting up with GitHub

Adding generate-tests workflow

Create .github/workflows/generate-tests.yml with the following content:

name: Generate Tests

on:
  pull_request:
    branches:
      - master
      - main

jobs:
  generate_tests:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 1
      - name: Generate and post tests
        run: npx ci-testgen generate-tests --gitRemote=github --token=${{ secrets.GITHUB_TOKEN }} --repoDir=$GITHUB_WORKSPACE --owner=${{ github.repository_owner }} --repo=${{ github.event.repository.name }} --pullRequestId=${{ github.event.number }}

Adding apply-tests workflow

Create .github/workflows/apply-tests.yml with the following content:

name: Apply Tests

on:
  issue_comment:
    types:
      - created
      - edited

jobs:
  apply-test:
    if: ${{ github.event.issue.pull_request && startsWith(github.event.comment.body, 'testgen') }}
    runs-on: ubuntu-latest
    permissions:
      contents: write
      pull-requests: write
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 1
      - name: Apply approved tests
        run: npx ci-testgen apply-tests --gitRemote=github --token=${{ secrets.GITHUB_TOKEN }} --repoDir='.' --owner=${{ github.repository_owner }} --repo=${{ github.event.repository.name }} --pullRequestId=${{ github.event.issue.number }} --triggerCommentId=${{ github.event.comment.id }} --triggerCommentBody=${{ github.event.comment.body }}

Done! You have completed the setup with GitHub.

Setting up with GitLab

Adding generate-tests workflow

Modify the .gitlab-ci.yml of the repo to add the new stage and job.

stages:
# ...existing stages...
  - generate_tests

# ...existing jobs...

generate_tests_job:
  stage: generate_tests
  needs: []
  image: node:20
  variables:
    GIT_DEPTH: 1
  script:
    - npx ci-testgen generate-tests --gitRemote=gitlab --token=$ACCESS_TOKEN --repoDir=$CI_PROJECT_DIR --projectId=$CI_PROJECT_ID --apiUrl=$CI_API_V4_URL --mergeRequestId=$CI_MERGE_REQUEST_IID
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'
      when: always

Adding apply-tests workflow

Since GitLab doesn't allow specific triggering of jobs, the following .gitlab-ci.yml should live inside an orphaned branch to ensure it only runs when explicitly triggered and doesn't interfere with other pipelines.

stages:
  - api_stage

api_only_job:
  stage: api_stage
  image: node:20
  rules:
    - if: $CI_PIPELINE_SOURCE == "trigger"
  variables:
    GIT_DEPTH: 1
  script:
    - npx ci-testgen apply-tests --gitRemote=gitlab --token=$ACCESS_TOKEN --repoDir=$CI_PROJECT_DIR --projectId=$CI_PROJECT_ID --apiUrl=$CI_API_V4_URL --triggerPayloadPath=$TRIGGER_PAYLOAD

To create an Orphaned Branch:

1. Create the branch

git checkout --orphan ci-testgen-apply-test-workflow
git rm -rf . # Clear the staging area

2. Add the above .gitlab-ci.yml file to the branch.

git add .gitlab-ci.yml

3. Commit and Push the Branch:

git commit -m "Add apply-tests workflow in orphaned branch"
git push origin ci-testgen-apply-test-workflow

Webhook Setup for GitLab

1. Provision a Pipeline Trigger Token:

   • Go to Settings > CI/CD > Pipeline Trigger Tokens.
   • Add a new token and call it Testgen Token.

2. Add a Webhook:

   • Go to Settings > Webhooks and click "Add new webhook".
   • For the URL, use: https://gitlab.com/api/v4/projects/<project-id>/ref/ci-testgen-apply-test-workflow/trigger/pipeline?token=<pipeline-trigger-token>. Replace <project-id> with your GitLab project ID.
   • Under 'Trigger' section, check "Comments".
   • Under 'Name', use: 'Apply Testgen Test' (optional)

CI/CD Variables Setup for GitLab

1. Provision an Access Token:

   • Go to Settings > Access Token and click "Add new token".
   • Use "Testgen" as the token name.
   • Select role to be developer and give the following scopes: api, read_api, read_repository, write_repository.
   • Set an appropriate expiration date for the token

2. Store the Access Token:

   • Go to Settings > CI/CD > Variables and store the token with the key as ACCESS_TOKEN.
   • Either choose "masked" or "masked and hidden".
   • Untick the "Protect variable" option.

Contributing

Check out our guide at CONTRIBUTING.md

Additional Resources

For more information, refer to the official GitHub and GitLab documentation.