@danielsimonjr/mathts-workbook

Headless runtime for MathTS scientific notebooks in the YAML-based .mtsw format. Load a workbook, run its cells in dependency order, and verify results — all from the terminal or programmatically.

Scope: This is the headless v1 — a CLI and runtime. Code cells evaluate MathTS expressions (via the sandboxed expression engine), not arbitrary TypeScript. A desktop GUI is a separate, later project that builds on this runtime.

Installation

npm install @danielsimonjr/mathts-workbook

CLI

# Run a workbook: executes cells in dependency order and prints per-cell results.
# Exits non-zero if any cell errors or any test assertion fails.
mtsw run example.mtsw
mtsw run example.mtsw -v        # also print the execution event stream
mtsw run example.mtsw --json    # machine-readable envelope on stdout
mtsw run example.mtsw -c gauss  # run one cell + its transitive deps (stateless)
mtsw run example.mtsw --timeout 5000   # run in a worker; killed (exit 1) if it exceeds 5s
mtsw run example.mtsw --expect-hash <sha256>  # refuse unless file SHA-256 matches

# Describe the structured document model (cells, outputs, dependency graph).
mtsw describe example.mtsw --json

# Validate structure: ids, dependency references, and cycles.
mtsw validate example.mtsw [--json]

# Print the dependency graph (human only; use `describe --json` for structured data).
mtsw graph example.mtsw                 # text adjacency
mtsw graph example.mtsw -f mermaid      # Mermaid `graph TD`
mtsw graph example.mtsw -f dot          # Graphviz `digraph deps { ... }`

# Engine introspection (for tooling / GUIs).
mtsw capabilities --json                # version, supported cell types, feature flags
mtsw templates --json                   # available `new` templates

# Scaffold a new workbook (<name>.mtsw) from a template.
mtsw new my-notebook                    # basic template; refuses to overwrite
mtsw new my-notebook -t basic --force   # overwrite an existing file

# Strip cell outputs (for git): prints to stdout, or -w to rewrite in place.
mtsw strip example.mtsw
mtsw strip example.mtsw -w

# Run and persist outputs back into the file (opt-in; never writes without --write).
mtsw run example.mtsw --write

# Edit cells (atomic, in-place; --json returns the updated doc, --dry-run previews).
mtsw cell add example.mtsw --type code --id gauss --content "n*(n+1)/2" --depends-on n
mtsw cell add example.mtsw --type code --id big --content-file body.txt   # or - for stdin
mtsw cell edit example.mtsw gauss --content "n*(n-1)/2"
mtsw cell move example.mtsw gauss --before n          # --before/--after <id> | --at <index>
mtsw cell rename example.mtsw n count                 # rewrites dependents' depends_on
mtsw cell rm example.mtsw n                           # refuses if cells depend on it
mtsw cell rm example.mtsw n --force                   # removes + detaches dependents

# Introspection + metadata.
mtsw functions --json                  # functions/constants cells can call (autocomplete)
mtsw meta get example.mtsw             # show workbook metadata
mtsw meta set example.mtsw --title "My Notebook" --author Ada --tags physics,demo

# Scaffold a workbook (templates: basic | empty | chart; -o writes to any path).
mtsw new notebook                        # notebook.mtsw in the CWD (basic template)
mtsw new draft --empty -o docs/draft.mtsw    # blank workbook at an explicit path
mtsw new demo -t chart                       # a line-chart example workbook

# Build a whole .mtsw from a JSON/YAML document (the inverse of `describe --json`).
echo '{"cells":[{"id":"a","type":"code","content":"6 * 7"}]}' | mtsw import -o out.mtsw
mtsw import doc.json --json               # validates (ids/deps/cycles); stdout if no -o

# Render to a self-contained HTML document (runs first, then renders).
mtsw export example.mtsw -o example.html     # one offline file; stdout if no -o
mtsw export example.mtsw --no-run            # render cached outputs without executing
mtsw export example.mtsw --format tex --fragment -o body.tex  # TeX body only (no preamble)
mtsw export example.mtsw --json              # envelope: { data: { path, bytes } }

# Other export formats: --format tex|pdf|json|ipynb (html is the default).
mtsw export example.mtsw --format ipynb -o example.ipynb   # Jupyter notebook (nbformat v4)

# Persistent session for a GUI/tooling: JSON-RPC 2.0 over stdio (NDJSON).
mtsw serve
# then JSON-RPC method "open" with params.path — the CLI ignores a file operand
#   -> {"jsonrpc":"2.0","id":1,"method":"open","params":{"path":"example.mtsw"}}
#   <- {"jsonrpc":"2.0","id":1,"result":{...describe doc...}}
#   -> {"jsonrpc":"2.0","id":2,"method":"run"}              # streams cell/event notifications
#   methods: open/describe/validate/graph/run/cell.*/meta.*/save/capabilities/functions/shutdown

export --format tex (programmatic). The same RenderDoc the CLI builds for TeX export is available as toTeX from the package entry (with RenderDoc, RenderCell, and ToTexOptions). A fragment omits the LaTeX preamble — the same shape as mtsw export … --format tex --fragment:

import { toTeX } from '@danielsimonjr/mathts-workbook';

const body = toTeX(
  { title: 'Notes', cells: [{ type: 'markdown', content: 'Energy $E=mc^2$' }] },
  { fragment: true }
);
// → markdown cell text only; no \\documentclass

Use buildDependencyGraph + toDOT when you need Graphviz DOT instead of Mermaid (mtsw graph -f dot).

export (self-contained HTML). mtsw export <file> --format html renders a notebook to a single offline .html with no external requests: markdown prose, equations typeset as MathML (rendered natively by modern browsers — Chromium ≥109, Firefox, Safari), code cells with their embedded outputs, ✓/✗ test badges, and (as of the chart slice) inline SVG plots. Per-node .toMathML() / .toTex() / .toHTML() and mathMLDocument come from @danielsimonjr/mathts-expression; the workbook runtime assembles full HTML with toHTML / toCSS / markdownToHtml / renderChart (workbook modules used by the CLI — not on the workbook package entry except mathMLToSVG). Equation cells contain MathTS expression syntax (e.g. c = 1 / sqrt(eps0 * mu0)), not raw LaTeX; they are display-only and rendered via toMathML. By default export runs the workbook first (use --no-run for cached outputs); a whole-run failure such as a dependency cycle fails loudly rather than emitting a misleading document.

export --format ipynb (Jupyter notebook). Renders the same run report to a structurally conformant nbformat v4 JSON document: markdown cells map to notebook markdown cells; every other cell type (code, equation, test, data, visualization) maps to a notebook code cell, with a computed result becoming an execute_result output (data['text/plain']), an error becoming an error output, and a chart becoming a display_data output (inline SVG). Shares the same run-then-render pipeline as --format html/tex (including --no-run, --json, -o).

run --timeout <ms> (kill-able worker-thread run). By default, cell execution runs in-process with no time budget — a runaway cell (e.g. an unbounded computation) hangs the process. --timeout runs the whole workbook in a worker_threads Worker and forcibly terminates it if it exceeds the budget, exiting 1 with a clear workbook execution exceeded <ms>ms and was terminated message; termination kills the worker outright, so it interrupts even a synchronous, CPU-bound runaway. It always runs the entire workbook to completion-or-termination, so it's incompatible with -c/-v. The same primitive is available programmatically as runWorkbookWithTimeout(source, { timeoutMs }) (throws WorkbookTimeoutError on timeout); cell outputs come back pre-formatted to strings (via formatResult), since engine class instances (Complex, matrices, …) don't survive the worker's postMessage.

serve (persistent session). One long-lived process holds the workbook in memory with a per-cell result cache and a stale set: a cell.* edit marks that cell and its transitive dependents stale, and a run re-executes only the stale cells (reusing cached outputs for the rest) — the incremental latency win a GUI needs. Requests are processed strictly in order; run streams cell/event notifications (flushed before that run's response in v1, not mid-run). Edits stay in memory until save. Single-document per process; concurrent writers are last-write-wins.

Editing notes. Cell edits are validity-preserving: an op that would create a duplicate/invalid id, a missing dependency, or a dependency cycle is rejected and the file is left byte-for-byte unchanged. Editing a cell clears its (now-stale) persisted output; --at N is the cell's final 0-based index; --force detaches dependents (clearing their outputs) but does not rewrite cell content, so a dependent that still references the removed id by name will error at run. Concurrent editors are last-write-wins in serve (last accepted cell.* op wins; use save to persist).

Diagnostics and errors are written to stderr; results (including --json) go to stdout, so the exit code can be used in scripts independently of the output.

Machine contract (--json). Every --json command emits one envelope on stdout: { schemaVersion: {major,minor}, command, ok, data, problems }. The envelope is emitted even on failure (and is cycle/BigInt-safe, so it never crashes on pathological data); the exit code mirrors ok for shells, but tooling/GUIs should read ok and treat a missing/unparseable envelope as the only transport error. Compatibility rule: ignore unknown fields when major matches; refuse on a major mismatch. run --cell <id> is stateless (it recomputes the target's transitive deps each call; run --cell --write persists only the executed cells). Stateful editing and incremental re-execution live in mtsw serve (see above); one-shot commands use this envelope only.

Saving / round-trip. serializeWorkbook (and the write commands above) round-trips a workbook through the parser: structure is preserved exactly, and persisted output values round-trip best-effort (plain numbers/strings/arrays/objects exactly; exotic types to their plain shape). All writes are atomic (temp file + rename). Note that any write path is parse→serialize and therefore drops YAML comments and re-orders keys — a CST-preserving in-place rewrite is a future enhancement.

Programmatic API

import { parseWorkbook, createExecutor, formatResult } from '@danielsimonjr/mathts-workbook';

const content = `
version: "1.0"
metadata:
  title: "My Workbook"
runtime:
  engine: mathts
  execution: reactive
cells:
  - code: "n * (n + 1) / 2"
    id: gaussSum
    depends_on: [n]
  - code: "10"
    id: n
  - test: "gaussSum == 55"
    id: checkGauss
    depends_on: [gaussSum]
`;

const result = parseWorkbook(content);
if (result.success && result.workbook) {
  const report = await createExecutor(result.workbook).runReport();
  for (const cell of report.cells) {
    console.log(cell.id, cell.status, formatResult(cell.output));
  }
  console.log('ok:', report.ok);
}

runReport() is continue-on-error and returns a structured RunResult (it never throws on a cell failure, and refuses a workbook with a dependency cycle). The older runAll() remains available as an event-stream API that throws on the first cell error.

Workbook format (.mtsw)

version: '1.0'
metadata:
  title: 'Example'
  author: 'Your Name'

runtime:
  engine: mathts
  execution: reactive # reactive | sequential | manual

cells:
  - markdown: |
      # Introduction
    id: intro

  - code: '{ pi: 3.14159, e: 2.71828 }'
    id: constants

  - test: 'constants.pi > 3.14'
    id: checkPi
    depends_on: [constants]

Each cell is a YAML mapping with exactly one type key (code, markdown, data, test, …) whose value is the cell content, plus:

• id (required) — must be a valid identifier ([A-Za-z_][A-Za-z0-9_]*); ids are how cells are referenced.
• depends_on (optional) — a list of cell ids this cell depends on.

Dependencies & scope

A dependency's result is injected into a cell's evaluation scope as a variable named by the dependency's id. To expose several values, return an object literal and read it with property access:

cells:
  - code: '{ n: 2, m: 3 }'
    id: pair
  - code: 'pair.n + pair.m' # -> 5
    id: total
    depends_on: [pair]

Scope is direct-only (non-transitive): a cell sees only the cells in its own depends_on, not their dependencies. To use a transitive value, list it explicitly.

Test cells

A test cell's expression must evaluate to a boolean: true passes, false fails, and a non-boolean result is reported as an error (use an explicit comparison). A failing test makes mtsw run exit non-zero — workbooks can verify themselves.

Cell types (v1)

| Type | Status | Description | | --------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------ | | markdown | ✅ | Documentation (passed through verbatim) | | code | ✅ | MathTS expression script; last value is the result | | data | ✅ | Structured YAML, parsed (hardened) into a value | | test | ✅ | Boolean assertion (true = pass) | | equation | ✅ | MathTS expression syntax; display-only (content echoed, not evaluated) | | visualization | ✅ | Chart/visual spec; display-only (content echoed, not evaluated) | | tensor | ⏳ | Deferred — parse fails: type 'tensor' is reserved but not yet supported (supported: code, markdown, data, test, equation, visualization) | | export | ⏳ | Same as tensor (export is reserved; not in the supported list) |

Execution modes

All modes execute cells in dependency order (graph.executionOrder). The only runtime difference is reactive, which emits cell:stale for dependents when a cell is edited or re-run. Sequential and manual use the same execution order without stale events (both still run the full graph on run / runReport).

Security

Code and test cells execute only through the MathTS sandboxed expression engine — no eval, Function, or vm. YAML (both the document and data-cell payloads) is parsed with a hardened core-schema configuration and a prototype-pollution guard.

License

MIT