sxy-test-runner

Mocha-like test runner for es modules (only), with watch mode, and re-running only tests related to changes (by dependency analysis), full async code support

import { describe } from 'sxy-test-runner'
import { expect } from 'chai'

function add(a, b) {
    return a + b
}

describe('The add() function', ({it}) => {
    it('should add 1 and 2, making 3', () => {
        const result = add(1, 2)
        expect(result).to.equal(3)
    })
})

Usage

1. Install

npm i -D sxy-test-runner

or

yarn add -D sxy-test-runner

or

pnpm add -D sxy-test-runner

2. Create a config (required)

sxy-test-runner init

sxy-test is an alias for sxy-test-runner and can be used instead

This will create a config in your project folder. It can be moved to a folder named "config" if you prefer, or specify a location using --config

3. Modify the config, specifying the base path for tests and code to watch, and glob patterns for tests and code to watch

{
    // base folder for tests matching and ignore patterns
    testsBase: 'dist',

    // glob pattern or array of glob patterns of test files
    tests: [
        '**/*.test.{js,jsx}'
    ],

    // watch mode configurations
    watch: {

        //// the base directory to watch files in
        watchFilesBase: 'dist',

        //// glob filter or array of global filters of files to watch
        watchFiles: '**/*.js',

    },
}

The other config options are fairly well explained in the config. More info on them later.

4. Run

npx sxy-test-runner

sxyt and sxy-test are aliases for sxy-test-runner and can be used instead

("npx" can be omitted in package.json scripts)

sxy-test-runner runs watch mode, re-running only relevant tests, by default.

While watch mode is idle, enter r or l to re-run the last set of tests, a to run all tests again, or f to re-run the currently failing tests.

Set watch.ttyActionsOnKeypress: true to trigger these commands as soon as their key is pressed, without waiting for Enter. It defaults to false.

To run tests only once and not watch (e.g. for ci/cd), use

npx sxy-test-runner once

To override the configured test pattern for either watch or once mode, use -t or --test-files. The pattern remains relative to testsBase:

npx sxy-test-runner once --test-files "**/*.focused.test.js"

To further constrain the configured test pattern without replacing it, use -f or --filter. This pattern is also relative to testsBase and works in both modes:

npx sxy-test-runner once --filter "**/users/**"

--test-files and --filter can be combined: the former replaces the configured test pattern, then the latter filters the discovered files.

--filter-describe and --filter-it filter describes and tests by name.

To specify an alternate config, use -c or --config

npx sxy-test-runner --config sxyt.alternate.config.js

Reporting for an agent

--agent replaces the usual (verbose) output with lines for key events - e.g. startup, failures, and completion.

npx sxy-test-runner once --agent

[sxyt] init | pid=16744
[sxyt] found | files=2
[sxyt] running | n=1 | files=2 | why=once | covers=all
[sxyt] fail | n=1 | file=tests/users.test.js | describe=creating a user | test=rejects a duplicate email | error=AssertionError: expected 409 to equal 200
[sxyt] done | n=1 | status=failed | desc=1 test failed | tests=2/3 | items=1/2 | files=1/2 | failing=1 | ms=150

Agent mode also sets the environment variables SXYT_AGENT and SXY_TEST_RUNNER_AGENT to true, so setup code and tests can tell they are running under it.

How it works

sxy-test-runner runs all tests inside one node process. Be careful with globals, which will be shared.

While tests are running, NODE_ENV defaults to test when it was not already set. SXYT and SXY_TEST_RUNNER are both set to true, allowing configuration and application code to detect sxy-test-runner specifically. In agent mode, SXYT_AGENT and SXY_TEST_RUNNER_AGENT are also set to true.

Caching modules

In watch mode modules will be reloaded as needed. With watch.dependencyEvaluation: false everything will be reloaded on each run.

To force a file never to be reloaded, insert ._cached_. into its filename, like:

import { database } from './fixtures/database._cached_.js'

Execution modes

execution settings let you choose whether to run test files, describe blocks, and individual tests sequentially or in parallel.

If working with globals you may want to use sequential only, to avoid conflicts on those globals.

Sequential mode also allows console log output to be displayed correctly with each test. Running in parallel mode does not allow log output to be matched up to each test, due to the use of a global override on console.log (other console output methods such as console.trace() are not covered currently)

If your code is isolated, and asynchronous, you can use parallel execution modes to radically speed up your tests, at the expense of the points mentioned above.

Dependencies

Dynamic imports will not be detected and will not trigger test re-runs.

To re-run every test on any change in the watched files instead set watch.dependencyEvaluation: false

Failing test re-running

Default behaviour is to re-run any failing test any time a change is made, even if that change isn't relevant to the failing test. This is to ensure you can see where attention is needed. Turn this off in the config if it is not convenient for you: config.watch.reRunFailingTests

Advanced usage

1. Run tasks at startup - setup

Edit config.setup. This config value can take a function, a file to import relative to the project folder, or an array or one or both of those.

2. Run tasks before each test file / describe / test globally

Edit config.beforeEachFile / config.beforeEachDescribe / config.beforeEachTest with a function, file or array of either.

If a file exports named values, or a function returns an object with properties, these keys will be added to a storage object (beware of conflicting names). This object is then passed to the afterEachFile / afterEachDescribe / afterEachTest tasks for cleanup.

    // tasks to run before each test file is run
    beforeEachFile: () => {
        const thing = prepSomething()
        return { thing }
    },

    // tasks to run after each test file is run
    afterEachFile: ({thing}) => {
        destroyingSomething(thing)
    },

3. Run code before each describe block, or at the end of a test file - per test file

Not yet implemented.

4. Run code before each test, after each test, or after the describe block - per describe block

import { describe } from 'sxy-test-runner'
import { expect } from 'chai'

describe('something', ({it, beforeEachTest, afterEachTest, afterDescribe}) => {

    // setup can take place here without any special measures
    // but you can only safely set globals for your test here if your
    // execution mode for files and describes is 'sequential' (default)
    const something = createSomething()

    // run this just before each test
    beforeEachTest(() => {
        prepSomething(something)
    })

    // run this just after each test
    afterEachTest(() => {
        clearupSomething(something)
    })

    it('should do something', () => {
        expect(something.didSomething()).to.equal(true)
    })
    it('should do another thing', () => {
        expect(something.didAnotherThing()).to.equal(true)
    })

    // teardown should go in afterDescribe.
    // tests are not run syncronously within the describe block (even sync tests)
    // so cleanup code here would run too early
    afterDescribe(() => {
        destroySomething(something)
    })

})

Exit codes for once

0:  success
20: test errors - a failed assertion, a test file that would not load,
    or a describe that threw
21: no test file matched the `tests` pattern
22: test files matched, but --filter-it/--filter-describe selected nothing
23: an option was not recognised, or one needing a value was given none
24: reserved for an error in sxy-test itself - not currently emitted

These start at 20 to stay clear of Node's reserved codes (1-14) and of signal exits (over 128), so a non-zero code is always sxy-test's own.

It could also crash due to exceptions without giving any of these codes, in which case Node supplies the code - usually 1 for an uncaught exception.