@akshatmittal/invoker

A strictly typed TypeScript DSL for matrix-driven regression workflows. Invoker expands Tasks into Vitest tests, runs each Task's Cases concurrently, and stores validated JSON Output in Vitest metadata for reporters and later analysis.

Install

pnpm add -D @akshatmittal/invoker vitest

Invoker supports Node 24.18.1 or newer within Node 24 and Vitest 4.1.10 or newer within Vitest 4.

Schedule GitHub Actions

@akshatmittal/invoker/github runs code-defined GitHub Actions schedules from a small, long-running Node process. It is independent from the Vitest SDK, so a scheduler-only installation does not need Vitest.

Create a GitHub App with repository Actions: read and write, disable its webhook, install it on the selected repositories, and generate a private key. No other repository, organization, user, OAuth, or webhook permissions are needed.

Install the scheduler with t3-env and Zod in a plain ESM application:

npm install @akshatmittal/invoker @t3-oss/env-core zod

// schedule.mjs
import { createEnv } from "@t3-oss/env-core";
import { defineGitHubSchedule } from "@akshatmittal/invoker/github";
import { z } from "zod";

const env = createEnv({
  server: {
    GITHUB_APP_ID: z.coerce.number().int().positive().max(Number.MAX_SAFE_INTEGER),
    GITHUB_APP_PRIVATE_KEY: z.string().min(1),
  },
  runtimeEnv: process.env,
});

await defineGitHubSchedule({
  app: {
    id: env.GITHUB_APP_ID,
    privateKey: env.GITHUB_APP_PRIVATE_KEY,
  },
  schedules: [
    {
      cron: "0 9 * * 1",
      timezone: "UTC",
      repository: "acme/regressions",
      workflow: "invoker.yml",
      ref: "main",
      inputs: { dataset: "weekly", publish: true },
    },
  ],
});

Schedules use five-field cron expressions and default to UTC when timezone is omitted. Configuration is fixed at startup. Every repository installation and active workflow is validated before timers begin; GitHub validates the ref, workflow_dispatch declaration, and input schema when a Dispatch is due.

Run exactly one replica. Dispatches may overlap, are not persisted or retried, and failures do not stop later occurrences. The process emits events through evlog's shared logger, so the host's filtering, sampling, redaction, and drain configuration applies. The module does not initialize or configure evlog. SIGINT and SIGTERM stop new Dispatches, await in-flight requests, and resolve the long-running promise.

Docker

Keep package.json, package-lock.json, and schedule.mjs in a deployment directory and build this image:

FROM node:24-slim

WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY schedule.mjs ./

USER node
CMD ["node", "schedule.mjs"]

Exclude .env, PEM, and private-key files from the build context. Inject GITHUB_APP_ID and the real multiline GITHUB_APP_PRIVATE_KEY through the runtime platform's secret mechanism; never bake them into the image. Use the platform's process-liveness check and restart policy with one replica. The scheduler intentionally has no HTTP health endpoint.

Define a Workflow

Declare a Task's required parameters once, then bind them from each Workflow. TypeScript checks the bindings while inferring the Task's Matrix, setup, and Output:

// regressions/model/tasks/evaluate-models.ts
import { defineTask } from "@akshatmittal/invoker";

type EvaluateParams = {
  environment: "staging" | "production";
  baseline: string;
};

export const evaluateModels = defineTask<EvaluateParams>()({
  name: "evaluate-models",
  matrix: async ({ params }) => ({
    model: await discoverModels(params.environment),
    dataset: ["support", "sales"],
  }),
  setup: async ({ params, cases }) => loadFixtures(params.environment, cases),
  run: async ({ params, matrix, setup, vitest }) => {
    vitest.expect(setup.has(matrix.dataset)).toBe(true);
    return {
      baseline: params.baseline,
      score: await evaluate(matrix, setup),
    };
  },
  teardown: async ({ setup }) => {
    await setup.close();
  },
});

// regressions/workflows/model-regressions.test.ts
import { defineWorkflow } from "@akshatmittal/invoker";
import { evaluateModels } from "../tasks/evaluate-models.js";

defineWorkflow({
  name: "model-regressions",
  metadata: { commit: process.env.GITHUB_SHA ?? "local" },
  matrix: async () => ({ environment: ["staging", "production"] }),
  tasks: ({ matrix }) => [
    evaluateModels({
      environment: matrix.environment,
      baseline: "2026-09-01",
    }),
  ],
});

Calling evaluateModels(...) binds inputs; it does not execute the Task. All parameter properties are required and JSON-compatible. Missing fields, optional parameter declarations, and incompatible bindings fail typechecking. There is no SDK defaults mechanism. Use setup for clients and other non-JSON resources.

For a parameterless Task, use defineTask()({ name, run }) and bind it with no argument. Workflow tasks is always a synchronous callback returning a nonempty Task list. It can select different Tasks per Workflow coordinate; names must be unique within each coordinate. Reuse the same Task in other Workflows with new bindings.

Both Matrix functions run during collection. The Workflow Matrix resolves once; Task binding runs once per Workflow coordinate, and each bound Task discovers its own Matrix once. Task discovery can run concurrently across coordinates. All bindings and matrices are validated before executable suites are registered; a discovery failure prevents that Workflow's execution.

Workflow axes multiply Task axes. For two environments and three Task models, there are six Cases. Execution follows the complete Task sequence for staging, then the complete sequence for production. Each bound Task gets its own setup and teardown, and its Cases run concurrently under Vitest's limits. Execution failures normally allow later Tasks to continue; Vitest's bail configuration controls stopping early.

Parameters are validated and snapshotted during collection and passed readonly to Matrix discovery, setup, run, and teardown. Task matrix and setup's cases contain only Task coordinates; Workflow coordinates enter through explicit parameter binding. Retries share that binding's parameter snapshot and setup result. Teardown runs once after successful setup.

Omitting either Matrix creates one coordinate, {}. Axis names must be non-empty, enumerable strings that are not array indexes. Empty axes and duplicate values are errors. Vitest shows Workflow → Workflow coordinate → Task → Case, with [1] for empty coordinates and numbered axis values otherwise.

Configure Vitest

The JSON reporter includes each Case's data at assertionResults[].meta.invoker:

// vitest.config.ts
import { defineConfig } from "vitest/config";

export default defineConfig({
  test: {
    maxConcurrency: 5,
    reporters: ["default", "json", ...(process.env.GITHUB_ACTIONS === "true" ? ["github-actions" as const] : [])],
    outputFile: {
      json: "./artifacts/invoker-results.json",
    },
  },
});

Run every Workflow or filter to one Task with ordinary Vitest commands:

pnpm vitest run
pnpm vitest run regressions/workflows/model-regressions.test.ts -t evaluate-models

Define additional Workflows in separate *.test.ts files. Vitest discovers them automatically; Invoker does not scan directories or require a central index.

Schema 2 keeps both coordinate scopes and all bound parameters in the JSON report:

{
  "schema": 2,
  "matrix": {
    "workflow": { "environment": "staging" },
    "task": { "model": "model-a", "dataset": "support" }
  },
  "params": { "environment": "staging", "baseline": "2026-09-01" },
  "metadata": { "commit": "abc123" },
  "output": { "baseline": "2026-09-01", "score": 0.92 }
}

matrix.workflow, matrix.task, and params are always present, with {} for empty scopes. Parameters are persisted in full, including fixed configuration; there is no automatic redaction. Static coordinates and parameters survive setup failures, skips, and retries. output is present only after a successful, JSON-valid Task return and is cleared before every retry. Vitest's report remains authoritative for status, failures, timing, hierarchy, and retries.

Notify Slack

Invoker's optional Slack reporter posts one Invoker Report parent message per Vitest run containing every Workflow card. Each card includes aggregate results, Workflow metadata, and a table with counts and durations per Workflow coordinate/Task pair. Failures, retries, and skips identify both Matrix scopes. Collection failures produce a failed Workflow card even without collected Cases. Full parameters are persisted in JSON rather than printed automatically in Slack. A shared footer contains the elapsed span from the first Case start to the final Case completion, a localized timestamp, and the optional run link. Final failures, successful retry details, skipped Case reasons, and unhandled run errors are posted in the same thread. The thread is limited to ten replies per run across all detail types. When details are omitted, the final reply directs readers to the run logs. Delivery failures are isolated to the affected reply. Ambiguous transport failures are not retried; an explicit Slack rate-limit rejection is reattempted only after its required delay.

import { slackReporter } from "@akshatmittal/invoker/slack";
import { defineConfig } from "vitest/config";

const runUrl =
  process.env.GITHUB_ACTIONS === "true"
    ? `${process.env.GITHUB_SERVER_URL}/${process.env.GITHUB_REPOSITORY}/actions/runs/${process.env.GITHUB_RUN_ID}`
    : undefined;

export default defineConfig({
  test: {
    reporters: [
      "tree",
      slackReporter({
        token: process.env.SLACK_BOT_TOKEN!,
        channel: process.env.SLACK_CHANNEL_ID!,
        runUrl,
      }),
    ],
  },
});

Create a Slack app with the chat:write bot scope, install it to the workspace, and expose its bot token and target channel ID as SLACK_BOT_TOKEN and SLACK_CHANNEL_ID. Invite the bot to private target channels. Slack delivery failures produce a warning but do not change Vitest's exit status.

GitHub Actions artifacts

Create the output directory before Vitest and upload the report even when the run fails:

- run: mkdir -p artifacts && pnpm vitest run
- if: always()
  uses: actions/upload-artifact@v4
  with:
    name: invoker-results
    path: artifacts/invoker-results.json

The artifact provides per-run retention and can be downloaded later for custom queries or reports. Invoker does not upload, index, or persist results itself.

v1 scope

Invoker does not provide a CLI, directory discovery, custom runner, general reporter framework, configuration helper, Task-level parallelism, matrix include/exclude, or hosted result storage. Use Vitest configuration and your CI runner for those concerns.