@speqkit/plugin-api

The only surface a plugin author sees. Types only — no runtime, no kernel internals, nothing that can drift.

Its major version is the compatibility boundary: the kernel refuses to load a plugin built against a different major, with a message rather than a crash halfway through a run. Adding is a minor. Changing or removing is a major.

import { definePlugin } from '@speqkit/plugin-api'

export default definePlugin({
  name: 'speqkit-plugin-example',
  setup(ctx) {
    ctx.defineStepType('example', {
      // What shape the input has …
      schema: { type: 'object', properties: { value: {} }, required: ['value'] },
      // … and whether it means anything. Both run before the suite does.
      validate: (step) => (step.value === '' ? [{ path: 'value', message: "'value' is empty" }] : []),
      execute: (exec, input) => ({ echoed: input.value })
    })
  }
})

A plugin never imports the kernel

This package is the only @speqkit/* a plugin may depend on, and it belongs in peerDependencies:

{
  "peerDependencies": { "@speqkit/plugin-api": "^0.4.0" },
  "devDependencies":  { "@speqkit/plugin-api": "^0.4.0" }
}

Everything a plugin needs from the running kernel arrives as ctx.host:

setup(ctx) {
  ctx.provide('report-mailer', {
    async rerun() {
      const tests = await ctx.host.discover({ tags: ['smoke'] })
      const outcome = await ctx.host.run(tests, { reporters: ['junit'] })
      return outcome.status
    }
  })
}

ctx.host is the session the plugin is already executing inside — not a way to start another, which is why there is no bootstrap on it. Reaching for speqkit instead costs two things, both of them silent: the installer sees a kernel in your dependencies and materialises a second copy of it into the store, and bootstrap() inside an already-booted process loads every plugin a second time into a registry nobody else can see. Whichever kernel then answers is decided by speq.lock, not by the speq the user installed.

The kernel and the plugin agree on the major of this package, checked as apiVersion when the plugin loads. That is the whole of the compatibility contract, and it is why a plugin needs no version of the kernel at all.

Stability

0.x — nothing here is stable. It changes without warning while the spine is still being proven against real plugins.

1.0 will freeze this package for the whole major, with at least a twelve month window on the previous one. Until that line is in this README, do not build anything you cannot afford to rewrite.

Changes

Numbered by what is on npm: 0.4.0, 0.9.0, 0.10.0, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 0.15.0. Everything below 0.4.0 was a change to the contract in this repository before anything was published from it, and the entry it landed under is kept as written.

0.15.0 — one addition, found by a CI log. PLUGIN_API_VERSION stays at 1.

• StepFailure and isStepFailure, and step-failed in STEP_CODES — how a step type says failed rather than error. execute() returned a value or threw, and a throw has always meant error: the harness never got an answer. That is right for a refused connection and wrong for every step that wraps other steps. loop, retry and use all learn that a child failed its assertions and all had to throw to report it, which relabelled "the system was wrong" as "the environment is broken" — @speqkit/plugin-loop carried the admission in a comment for two releases.

  The line is not cosmetic. @speqkit/plugin-gate routes every red test across it to decide whether to fix the code or the stand, JUnit puts <failure> on one side and <error> on the other, and a build that reports errored sends its reader to look at the network. A plugin with no opinion keeps throwing ordinary errors and keeps getting error: claiming the system was wrong is the stronger statement, so it is the one that has to be made deliberately.

  Matched by a marker field rather than instanceof, because a plugin that bundled its own copy of this package would otherwise throw something the kernel's class does not recognise — and the failure would silently become an error again, which is the bug the type exists to fix.

0.14.0 — two additions, both of them found by a suite rather than by reading this file. PLUGIN_API_VERSION stays at 1.

• The path language: pathSegments, readSegments, readPath and PathRead — the first runtime this package has offered beyond definePlugin, and it is here because a path is not any one plugin's business. ${…} reads one, every path: an assertion offers reads one, and a plugin of your own taking a path: field should read the same one. It was written out twice before this — the kernel's copy and @speqkit/plugin-assert's — and the two had already drifted over whether a segment may carry whitespace, which is a path meaning two things depending on who was asked. The grammar gains [*], every element, with the rest of the path applying to each; it is a wildcard and takes no condition, because a path that can carry a predicate is a language speq validate cannot check.
• ExecContext.check — a step asks the loaded assertion vocabulary whether a value satisfies some clauses, and gets the answers instead of recording them. pick in @speqkit/plugin-data needed to say the item with a required option group, and the alternative was a second list of comparison words living in that plugin. Two lists of the same words diverge — one of them gets at_least and the other does not. There is one list, defineAssertionType is how it grows, and a check somebody else published works as a filter clause the day it is installed. It records nothing, emits no assertion.evaluated, does not resolve ${…} a second time, and throws on a clause type nothing provides rather than answering passed: false.

0.13.0 — six additions, all optional, and every one of them a thing a caller could previously only get at by reading prose or keeping state of its own. PLUGIN_API_VERSION stays at 1.

• STEP_CODES, ASSERTION_CODES, TEST_CODES, and a code on StepRecord, AssertOutcome, TestOutcome and the events — why something did not pass, in a word a program may match on. message is written for a person and may be reworded in any release; these may not. The kernel sets them and a plugin never does: a vocabulary a plugin could add to is a vocabulary nothing downstream can switch on.
• StepDef.when — run this step only if the value is true. A field of the spine because the decision happens before the step's type is looked up, so a plugin could not own it: a step whose plugin is not loaded here can still be switched off. A template or a literal; there is no expression language.
• TestDef.timeout — the longest a whole test may take, covering the givens, setup and the body. cleanup gets a fresh budget, because a test that ran out of time is the one that left something behind.
• Diagnostic.level — absent means error, which is what every diagnostic was. warn is for something legal that is probably not what was meant, and it exists for exactly one shape of that: a key filed under meta whose name reads like behaviour.
• ValueContext, on ValueProviderDef.resolve — which test is asking, said by the kernel at the moment it asks. A provider that had to answer per test kept a "current test" from a test:before hook, which is adjacency; under --workers 4 it held whichever suite started last, and plugin-data handed one test another's supposedly unique value.

0.12.0 — StepTypeDef gained an optional binds(step): the names a nesting step makes addressable to the steps under it, beyond what is visible outside — loop answers [as, as + 'Index'], retry answers nothing. What it buys is speq validate reading every ${…} before the run. The kernel knows what a test binds itself — its givens in order, the id of every step above — and reports a reference to none of them as unresolved-reference, forward-reference or unknown-provider; what a nesting step binds it cannot know, so a step type with steps and no binds is taken at its word and nothing is reported under it. An added optional member, so every 0.11.0 plugin still satisfies the contract and PLUGIN_API_VERSION stays at 1.

Also in 0.12.0: InputSchema says what the kernel reads of it. It had been "JSON-Schema-shaped" since the first commit and the kernel read two words — required, and additionalProperties: false — so method: GETT, attempts: "3" and a typo one level down in retry: all went out on the wire. The kernel now reads type, enum, const, nested properties, items, the bounds, pattern and the three combinators, at every depth; a value that is still a whole ${…} template fits any shape. The type gained description, enum, a type that may be a list and an additionalProperties that may be a schema — every one of them already legal under the index signature, now named. And configSchema is read: a plugin's block in speq.yaml that does not match it is a startup refusal, invalid-plugin-config. ctx.config() had said "already validated" for as long as it existed, and nothing had ever validated it.

0.11.0 — Host gained capabilities(), and with it Capabilities and Capability: every step type, assertion, value provider, reporter and loader the loaded plugins define, with the InputSchema each declared. The schemas had been in the registry since the plugin that owns them registered and could not be reached from outside the process, so an editor offering completion, a palette in a panel and a system prompt describing speq to a model each carried a copy of the vocabulary — one that goes stale the moment somebody installs a plugin, and goes stale silently, because a suite written against the wrong vocabulary looks exactly like a suite with a typo in it.

Also in 0.11.0: PluginSpec gained docs, and every contribution def gained summary. A plugin could declare its whole grammar and not one word about what any of it is for — so what it is for lived in a README on a website, which is a document a session cannot ask, cannot check, and which is wrong the moment somebody renames a step type. speq docs answers out of what is declared here, and speq docs --check fails on an example naming a capability that no longer exists. It is optional on the type on purpose: a fixture plugin declared inside a test has no documentation and should not have to say so. The obligation lands where a package meets a registry, in check-plugin-package.mjs.

Also in 0.11.0: test.started gained suite. A test's suite was said only by the bracketing — the last suite.started before it — and the bracketing is adjacency, which G4 takes away the moment two suites run at once. Two reporters in the box were reading it that way, one of them after the same fault had already been fixed in the other. A test now carries its own answer, and G6 says so.

Also in 0.11.0: test.started and TestOutcome gained tags. A reporter could group by suite, by file and by meta, and not by the label the run was actually selected with, so anything reporting per ticket or per component had to re-discover the project to learn what it had just watched run. It is the effective set, suites and cases included — the one --tags filters on. Found by writing @speqkit/plugin-gate against the published contract, which is the fourth hole a plugin has found that nobody inside the kernel could see.

Also in 0.11.0: ExecContext gained record(detail), StepRecord gained detail, and step.finished carries it. A step's result never entered the event stream, so no reporter could print the request and the response of a step that failed, whatever flag it was given — the only way to see an exchange was to run the test again with a proxy in front of it. The whole result on every step.finished was the other way to close that, and it makes the run log as large as every response body in the run, nearly all of them from steps that passed. So the step decides what is worth recording and the kernel decides whether it is worth keeping: dropped when the step passed, carried when it did not. Recording before the work rather than after it is deliberate and is what a callback over the result could not do — a request that never comes back has no result to describe it, and the step said what it was attempting before it went quiet.

Also in 0.11.0: Diagnostic gained a required code and ValidationProblem an optional one. The message is written for a person and may be reworded in any release; the code is written for a program and may not — it is what lets a caller tell a step type that does not exist from one whose input is malformed without matching substrings of coloured output. The kernel's codes are bare words; whatever a plugin's own validate returns is prefixed by the kernel with that plugin's short name — http/unknown-topic, or http/invalid when the plugin named none — so a plugin can add codes for as long as it likes and never collide with one the kernel means to use. Added members on things the kernel produces, so every 0.10.0 plugin still satisfies the contract and PLUGIN_API_VERSION stays at 1.

0.10.0 — SuiteDef and CaseDef; LoaderDef.suiteFiles and LoaderDef.loadSuite, so a loader can declare suites without reimplementing the tree, the identity or the inheritance; TestDef.cases, TestDef.group and TestDef.suites; RunRequest.concurrency; DiscoverQuery.names; TestOutcome.group; ValidateContext.suite. suite.started gained parent, title and pending, and test.started gained group. No ninth contribution point was opened — a suite was tried as a plugin first, and the experiment came back no: grouping, identity and inheritance are all settled before any hook fires. PLUGIN_API_VERSION stays at 1.

0.9.0 — StepDef.assert, TestDef.setup and TestDef.cleanup; TestDef.variables, resolved one at a time in declaration order; meta, which the kernel carries and never reads; and pending, which takes a reason rather than a flag. Four gaps in the model, every one of them closed by a field on the spine rather than by a new contribution point.

Also in 0.9.0: StepTypeDef and AssertionTypeDef gained an optional validate, and with it Validator, ValidateContext and ValidationProblem. A schema settles the shape of an input and nothing more, so anything a plugin knows about an input — that the schema file an assertion names is on disk, that over and times exclude each other, that a topic is one of the configured ones — had nowhere to be said but the middle of a run, from a step type that cannot name the file the mistake is in. The kernel keeps the walk and the addressing: a plugin returns messages and, at most, a path inside its own step. Synchronous on purpose — validation runs in front of every run and is expected to cost milliseconds, so reading a file is fine and a network call is not; a validator that throws is reported as a bug in the plugin and the rest of the diagnostics still come back. Optional members on existing interfaces, so every 0.4.0 plugin still satisfies the contract and PLUGIN_API_VERSION stays at 1.

And in 0.9.0, with no change to any type: ValueProviderDef.resolve has always been declared as unknown | Promise<unknown>, and the kernel now honours it. It used to drop the Promise itself into the request body, silently. A provider is asked once per resolution pass and every key a step needs is awaited at once; ExecContext.resolve stays synchronous, and throws where a template it is handed names an asynchronous provider.

0.4.0 — PluginContext gained host, and with it Host, Diagnostic, ArtifactRecord, RunOutcome, TestOutcome, RecordedRun, DiscoverQuery and RunRequest. A plugin that needed to discover, validate, run or replay had no way to ask the kernel and so imported speqkit — which put the kernel in the plugin's published dependencies and had the installer place a second copy of it in the store, while the plugin called bootstrap() inside a process that had already booted one. Added members only, so every 0.3.0 plugin still satisfies the contract and PLUGIN_API_VERSION stays at 1.

0.3.0 — ReporterDef gained an optional init(ctx) carrying runId, outputDir and runDir. A reporter that writes a file could not previously learn where to write it: the run's directory does not exist until the run starts. Optional method on an existing interface, so every 0.2.0 reporter still satisfies it, and PLUGIN_API_VERSION stays at 1.

0.2.0 — artifact.attached gained path, set when the run wrote the body somewhere. Added field, so plugins built against 0.1.0 keep working and PLUGIN_API_VERSION stays at 1. This is the versioning rule doing its job on its first real occasion, rather than a rule we only wrote down.