wstp-backend

Author: Nikolay Gromov

Native Node.js addon for Wolfram kernel communication over WSTP. Enables notebook-style evaluation, real-time streaming, and robust kernel management from JavaScript.

Purpose

This package lets you control a WolframKernel process from Node.js, supporting both batch and interactive notebook-style evaluation. It is ideal for building VS Code extensions, automating Mathematica workflows, and integrating Wolfram computation into JS apps.

Installation

Prerequisites

| Requirement | Notes | |-------------|-------| | macOS | Tested on macOS 13+ (ARM64 and x86-64) | | Windows 10/11 x64 | See InstallationWindows.md for the full Windows guide | | Linux x86-64 / ARM64 | Should work with standard node-gyp toolchain | | Node.js ≥ 18 | Earlier versions may work but are untested | | Clang / Xcode Command Line Tools (macOS) | xcode-select --install | | MSVC Build Tools with C++ workload (Windows) | Visual Studio 2019+ or Build Tools | | Wolfram Mathematica or Wolfram Engine | Provides WolframKernel and the WSTP SDK headers/libraries |

1. Clone

git clone https://github.com/vanbaalon/mathematica-wstp-node.git
cd mathematica-wstp-node

2. Install Node dependencies

npm install

3. Build the native addon

bash build.sh

Quick Start

const { WstpSession } = require('./build/Release/wstp.node');
const session = new WstpSession();
session.evaluate('Prime[10]').then(r => {
  console.log(r.result.value); // 29
  session.close();
});

API Reference

See API.md for full API details, usage examples, and advanced features.

• Priority sub() calls
• Abort a long computation
• Dialog subsessions
• Variable monitor — peeking at a running loop
• Real-time side channel (WstpReader)
• Parallel independent kernels

6. Error handling
7. Diagnostic logging

Installation

Prerequisites

| Requirement | Notes | |-------------|-------| | macOS | Tested on macOS 13+ (ARM64 and x86-64) | | Windows 10/11 x64 | See InstallationWindows.md for the full Windows guide | | Linux x86-64 / ARM64 | Should work with standard node-gyp toolchain | | Node.js ≥ 18 | Earlier versions may work but are untested | | Clang / Xcode Command Line Tools (macOS) | xcode-select --install | | MSVC Build Tools with C++ workload (Windows) | Visual Studio 2019+ or Build Tools | | Wolfram Mathematica or Wolfram Engine | Provides WolframKernel and the WSTP SDK headers/libraries |

Windows users: follow InstallationWindows.md instead of the steps below.

1. Clone

git clone https://github.com/vanbaalon/mathematica-wstp-node.git
cd mathematica-wstp-node

2. Install Node dependencies

npm install

This pulls in node-addon-api and node-gyp (used by the build script).

3. Compile the native addon

bash build.sh

Output: build/Release/wstp.node

The script automatically locates the WSTP SDK inside the default Wolfram installation (/Applications/Wolfram 3.app/...). If your Wolfram is installed elsewhere, edit the WSTP_INC and WSTP_LIB variables at the top of build.sh.

4. Run the test suite

node test.js

Expected last line: All 61 tests passed.

A more comprehensive suite (both modes + In/Out + comparison) lives in tmp/tests_all.js:

node tmp/tests_all.js

Expected last line: All 41 tests passed.

5. Quick smoke test

const { WstpSession } = require('./build/Release/wstp.node');

(async () => {
  const session = new WstpSession();
  const r = await session.evaluate('Prime[10]');
  console.log(r.result.value);   // 29
  console.log(r.cellIndex);      // 1
  console.log(r.outputName);     // "Out[1]="  (may vary by session)
  session.close();
})();

Default kernel path (macOS): /Applications/Wolfram 3.app/Contents/MacOS/WolframKernel

Pass an explicit path as the first argument to new WstpSession(path) if yours differs.

Batch mode vs interactive mode

The session constructor accepts an optional second argument:

// Batch mode (default) — EvaluatePacket, bypasses kernel main loop
const session = new WstpSession(kernelPath);
// or explicitly:
const session = new WstpSession(kernelPath, { interactive: false });

// Interactive mode — EnterExpressionPacket, full notebook-style evaluation
const session = new WstpSession(kernelPath, { interactive: true });

Batch mode (interactive: false, default)

Uses EvaluatePacket which bypasses the kernel's main evaluation loop. Fast and lightweight — the kernel evaluates the expression and returns the result without touching In[n]/Out[n]/$Line.

• In[n], Out[n], %, %% are not populated by evaluations
• $Line stays at 1 regardless of how many evaluations are run
• outputName in EvalResult is always "" in steady state
• Suitable for scripting and batch processing where history is not needed

Interactive mode (interactive: true)

Uses EnterExpressionPacket which runs each evaluation through the kernel's full main loop — exactly as a Mathematica notebook does.

• In[n], Out[n], %, %% all work and persist across evaluations
• $Line increments with every evaluation
• cellIndex in EvalResult reflects the actual kernel $Line (not a JS counter)
• outputName in EvalResult is "Out[n]=" for non-Null results, "" for suppressed
• $HistoryLength controls memory usage (default 100)
• Suitable for notebook-like workflows and sessions that rely on running history

Return Types

WExpr

A Wolfram Language expression as a plain JS object. The type field determines the shape:

| type | Additional fields | Example | |--------|------------------|---------| | "integer" | value: number | { type: "integer", value: 42 } | | "real" | value: number | { type: "real", value: 3.14 } | | "string" | value: string | { type: "string", value: "hello" } | | "symbol" | value: string | { type: "symbol", value: "Pi" } | | "function" | head: string, args: WExpr[] | { type: "function", head: "Plus", args: [{type:"integer",value:1}, {type:"symbol",value:"x"}] } | | (absent) | error: string | internal error — normally never seen |

Symbols are returned with their context stripped: System\Pi→"Pi"`.

EvalResult

The full result of one evaluate() call:

{
  cellIndex:  number;   // Batch: JS-tracked counter (1-based per session).
                        // Interactive: kernel $Line at time of evaluation.
  outputName: string;   // Interactive: "Out[n]=" for non-Null results; "" for suppressed.
                        // Batch: derived from kernel OUTPUTNAMEPKT if sent.
  result:     WExpr;    // the expression returned by the kernel
  print:      string[]; // lines written by Print[], EchoFunction[], etc.
  messages:   string[]; // kernel messages, e.g. "Power::infy: Infinite expression..."
  aborted:    boolean;  // true when the evaluation was stopped by abort()
}

WstpSession

Constructor

const session = new WstpSession(kernelPath?, options?);

Launches a WolframKernel process, connects over WSTP, and verifies that $Output routing is working. Consecutive kernel launches occasionally start with broken output routing (a WSTP quirk); the constructor detects this automatically and retries up to 3 times, so it is transparent to callers.

| Parameter | Type | Default | |-----------|------|---------| | kernelPath | string | /Applications/Wolfram 3.app/Contents/MacOS/WolframKernel | | options.interactive | boolean | false — see Batch mode vs interactive mode |

Throws if the kernel cannot be launched or the WSTP link fails to activate.

evaluate(expr, opts?)

session.evaluate(expr: string, opts?: EvalOptions): Promise<EvalResult>

Evaluate expr in the kernel and return the full result.

expr is passed to ToExpression[] on the kernel side, so it must be valid Wolfram Language syntax. Multiple concurrent calls are serialised through an internal queue — it is safe to fire them without awaiting.

One call = one cell. Newlines and semicolons inside expr do not split it into multiple evaluations; the kernel sees them as a single CompoundExpression and returns only the last value. Use separate evaluate() calls to get separate cellIndex / outputName values. A trailing semicolon suppresses the return value (the kernel returns Null and outputName will be "").

opts fields (all optional):

| Option | Type | Description | |--------|------|-------------| | onPrint(line: string) | callback | Each Print[] or similar output line, as it arrives | | onMessage(msg: string) | callback | Each kernel warning/error, as it arrives | | onDialogBegin(level: number) | callback | When Dialog[] opens | | onDialogPrint(line: string) | callback | Print[] output inside a dialog | | onDialogEnd(level: number) | callback | When the dialog closes | | interactive | boolean | Per-call override of the session's interactive mode. true forces EnterExpressionPacket (populates In/Out); false forces EvaluatePacket (batch, no history). Omit to use the session default set in the constructor. |

All callbacks fire on the JS main thread before the Promise resolves. The Promise is guaranteed not to resolve until all queued callback deliveries have completed.

const r = await session.evaluate('Do[Print[i]; Pause[0.3], {i,1,4}]', {
    onPrint: (line) => console.log('live:', line),  // fires 4 times during eval
});
// r.print is also ['1','2','3','4'] — same data, delivered after eval completes

// Session is interactive (default), but force this one call to be batch (no Out/In side-effects):
const internal = await session.evaluate('VsCodeRenderNth[1, "SVG", 1.0]', { interactive: false });

// Session is batch (default), but force this one call to go through the main loop:
const r2 = await session.evaluate('x = 42', { interactive: true });
console.log(r2.outputName);  // "Out[1]="
console.log(r2.cellIndex);   // 1

sub(expr)

session.sub(expr: string): Promise<WExpr>

Lightweight evaluation that resolves with just the result WExpr (no cell index, no print/messages arrays).

sub() has higher priority than evaluate(): it always runs before the next queued evaluate(), regardless of arrival order. If the session is currently busy, sub() waits for the in-flight evaluation to finish, then runs ahead of all other queued evaluate() calls. Multiple sub() calls are ordered FIFO among themselves.

const pid  = await session.sub('$ProcessID');   // { type: 'integer', value: 12345 }
const info = await session.sub('$Version');     // { type: 'string', value: '14.1 ...' }

abort()

session.abort(): boolean

Interrupt the currently-running evaluate() call. Thread-safe — safe to call from any callback or timer.

The in-flight evaluate() Promise resolves (not rejects) with:

{ aborted: true, result: { type: 'symbol', value: '$Aborted' }, ... }

The kernel remains alive after abort. Subsequent evaluate() and sub() calls work normally.

const p = session.evaluate('Do[Pause[0.1], {1000}]');  // ~100 s computation
await sleep(500);
session.abort();       // cancels after ~500 ms
const r = await p;
// r.aborted === true
// r.result  === { type: 'symbol', value: '$Aborted' }

closeAllDialogs()

session.closeAllDialogs(): boolean

Unconditionally reset all dialog state on the Node.js side.

• Drains the internal dialog queue, immediately rejecting every pending dialogEval() and exitDialog() promise with an error — no callers are left waiting forever.
• Clears isDialogOpen to false.

This does not send any packet to the kernel — it only fixes Node-side bookkeeping. Use it in error-recovery paths, before abort(), or whenever you need to guarantee clean dialog state without knowing whether a dialog is actually still running.

Returns true if isDialogOpen was true before the call (something was cleaned up), false if it was already clear.

// Safe no-op when there is no open dialog:
const cleaned = session.closeAllDialogs();  // false

// Reliable recovery pattern before abort:
session.closeAllDialogs();  // reject any hanging dialog promises immediately
session.abort();

// Queued dialogEval() promises reject with a descriptive error:
const p = session.evaluate('Dialog[]');
await pollUntil(() => session.isDialogOpen);
const pe = session.dialogEval('1 + 1').catch(e => e.message);
session.closeAllDialogs();  // pe rejects → "dialog closed by closeAllDialogs"
session.abort();

dialogEval(expr)

session.dialogEval(expr: string): Promise<WExpr>

Evaluate expr inside the currently-open Dialog[] subsession. Rejects immediately if isDialogOpen is false.

Returns just the WExpr result, not a full EvalResult.

Important: kernel global state persists into and out of a dialog. Variables set before Dialog[] are in scope inside; mutations made with dialogEval() persist after the dialog closes.

const p = session.evaluate('x = 10; Dialog[]; x^2');
await pollUntil(() => session.isDialogOpen);

const xVal = await session.dialogEval('x');      // { value: 10 }
await session.dialogEval('x = 99');              // mutates kernel state
await session.exitDialog();

const r = await p;  // r.result.value === 9801  (99^2)

exitDialog(retVal?)

session.exitDialog(retVal?: string): Promise<null>

Close the currently-open Dialog[] subsession.

Sends EnterTextPacket["Return[retVal]"] — the interactive-REPL packet that the kernel recognises as "exit the dialog". This is not the same as dialogEval('Return[]'), which uses EvaluatePacket and leaves Return[] unevaluated.

Resolves with null when EndDialogPacket is received. Rejects immediately if isDialogOpen is false.

| Call | Effect | |------|--------| | exitDialog() | Dialog[] evaluates to Null | | exitDialog('42') | Dialog[] evaluates to 42 | | exitDialog('myVar') | Dialog[] evaluates to the current value of myVar |

// Pattern: open dialog, interact, close with a return value
const p = session.evaluate('result = Dialog[]; result * 2');
await pollUntil(() => session.isDialogOpen);
await session.dialogEval('Print["inside the dialog"]');
await session.exitDialog('21');
const r = await p;  // r.result.value === 42

interrupt()

session.interrupt(): boolean

Send WSInterruptMessage to the kernel (best-effort). The C++ backend handles the kernel's MENUPKT interrupt-menu response automatically by reading the type+prompt payload and replying with bare string "i" (inspect mode), causing the kernel to open a Dialog[] subsession. isDialogOpen will flip to true when BEGINDLGPKT arrives.

The evaluated expression must have been started with onDialogBegin / onDialogEnd callbacks for the dialog to be serviced correctly.

// interrupt() — no Wolfram-side handler needed (C++ handles MENUPKT automatically)
// The evaluate() call must include dialog callbacks:
const mainEval = session.evaluate('Do[Pause[0.1], {1000}]', {
    onDialogBegin: () => {},
    onDialogEnd:   () => {},
});
session.interrupt();
await pollUntil(() => session.isDialogOpen);
const val = await session.dialogEval('$Line');
await session.exitDialog();
await mainEval;

Optionally, install a Wolfram-side handler to bypass MENUPKT entirely:

await session.evaluate('Internal`AddHandler["Interrupt", Function[Null, Dialog[]]]');

createSubsession(kernelPath?)

session.createSubsession(kernelPath?: string): WstpSession

Launch a completely independent kernel as a new WstpSession. The child has isolated state (variables, definitions, memory) and must be closed with child.close().

close()

session.close(): void

Terminate the kernel process, close the WSTP link, and free all resources. Idempotent — safe to call multiple times. After close(), calls to evaluate() reject immediately with "Session is closed".

isOpen / isDialogOpen / kernelState

session.isOpen:       boolean  // true while the link is open and the kernel is running
session.isDialogOpen: boolean  // true while inside a Dialog[] subsession
session.kernelState:  string   // multi-dimensional state snapshot

kernelState returns a space-separated dimension=value string covering five independent state dimensions that can be active in parallel:

| Dimension | Values | Description | |------------|---------------------------------------|-------------| | activity | Idle, Eval, SubIdle, WhenIdle | Main job the kernel is doing | | dialog | None, UserDialog, DynDialog | Dialog subsession type | | sub | None, DynExpr, SubBusy | Sub-work inside a dialog | | abort | None, Aborting | Abort in progress | | link | Alive, Dead | WSTP link health |

Example:

console.log(session.kernelState);
// "activity=Eval dialog=DynDialog sub=DynExpr abort=None link=Alive"

Every transition is logged via setDiagHandler with a [State:<dim>] category tag (e.g. [State:activity] Idle -> Eval (MaybeStartNext:eval)). Internal safeguards use these dimensions to prevent dangerous operations — e.g. blocking WSInterruptMessage when the link is dead or an abort is already in progress.

Dynamic eval API

Register Wolfram Language expressions for automatic periodic evaluation during cell computations. A RunScheduledTask in the kernel periodically calls Dialog[]; the C++ layer intercepts each BEGINDLGPKT, evaluates all registered expressions inline, stores the results, and closes the dialog.

| Method | Description | |--------|-------------| | registerDynamic(id, expr) | Register (or upsert) an expression for periodic evaluation | | unregisterDynamic(id) | Remove one registration by id | | clearDynamicRegistry() | Remove all registrations and clear results buffer | | getDynamicResults() | Return Record<string, DynResult> and clear buffer | | setDynamicInterval(ms) | Set the ScheduledTask period (0 = off) | | setDynAutoMode(auto) | true (default) = C++ handles dialogs; false = legacy JS path | | dynamicActive | Read-only: true when registry is non-empty and interval > 0 |

interface DynResult {
    value:      string;  // string-form result
    timestamp:  number;  // Unix timestamp (ms)
    error?:     string;  // set if evaluation failed
}

See API.md for full documentation.

WstpReader

A reader that connects to a named WSTP link created by the kernel (via LinkCreate) and receives expressions pushed from the kernel side (via LinkWrite).

Use this when you need real-time data from the kernel while the main link is blocked on a long evaluation.

Constructor

const reader = new WstpReader(linkName, protocol?);

| Parameter | Type | Default | |-----------|------|---------| | linkName | string | (required) — value of linkObject[[1]] or LinkName[linkObject] on the Wolfram side | | protocol | string | "TCPIP" |

Throws if the connection fails. The WSTP handshake (WSActivate) is deferred to the first readNext() call, so the constructor never blocks the JS main thread.

readNext()

reader.readNext(): Promise<WExpr>

Block (on the thread pool) until the kernel writes the next expression with LinkWrite. Resolves with the expression as a WExpr.

Rejects when the kernel closes the link (LinkClose[link]) or the link encounters an error.

close() / isOpen

reader.close(): void
reader.isOpen:  boolean

Full pattern

(* Wolfram side — create a push link *)
$mon = LinkCreate[LinkProtocol -> "TCPIP"];
linkName = $mon[[1]];   (* share this string with the JS side somehow *)

(* Write immediately, then pause (not: pause then write) *)
Do[
    LinkWrite[$mon, {i, randomVal}];
    Pause[0.5],
    {i, 1, 20}
];
Pause[1];               (* give reader time to drain final value *)
LinkClose[$mon];

// JS side — connect and read
const reader = new WstpReader(linkName, 'TCPIP');
try {
    while (reader.isOpen) {
        const v = await reader.readNext();
        console.log('received:', JSON.stringify(v));
    }
} catch (e) {
    if (!e.message.includes('closed')) throw e;  // normal link-close rejection
} finally {
    reader.close();
}

Timing rules for reliable delivery:

• Call LinkWrite[link, expr] before any Pause[] after each value. A Pause before LinkWrite can cause the reader to block inside WSGetType, which then races with the simultaneous LinkClose on the last value.
• Add Pause[1] before LinkClose so the reader receives the final expression before the link-close signal arrives.

setDiagHandler(fn)

setDiagHandler(fn: ((msg: string) => void) | null | undefined): void

Register a JS callback that receives internal diagnostic messages from the C++ layer. The callback fires on the JS main thread. Pass null to clear.

Messages cover:

• [Session] — kernel launch, restart attempts, WarmUp results
• [WarmUp] — per-attempt $WARMUP$ probe
• [Eval] pkt=N — every WSTP packet in the evaluation drain loop
• [TSFN][onPrint] dispatch +Nms "..." — TSFN call timestamp (compare with your JS callback timestamp to measure delivery latency)
• [WstpReader] — WSActivate, spin-wait trace, ReadExprRaw result

setDiagHandler((msg) => {
    const ts = new Date().toISOString().slice(11, 23);
    process.stderr.write(`[diag ${ts}] ${msg}\n`);
});

// Disable:
setDiagHandler(null);

Alternative — set DEBUG_WSTP=1 in the environment to write the same messages directly to stderr (no JS handler needed, useful in scripts):

DEBUG_WSTP=1 node compute.js 2>diag.txt

Usage Examples

Basic evaluation

const { WstpSession } = require('./build/Release/wstp.node');

const KERNEL = '/Applications/Wolfram 3.app/Contents/MacOS/WolframKernel';
const session = new WstpSession(KERNEL);

// Simple expression
const r = await session.evaluate('Expand[(a + b)^4]');
console.log(r.result);
// { type: 'function', head: 'Plus', args: [ ... ] }

// Integer result
const n = await session.evaluate('Prime[100]');
console.log(n.result.value);  // 541

// String result
const v = await session.evaluate('"Hello, " <> "World"');
console.log(v.result.value);  // "Hello, World"

session.close();

Interactive mode — In/Out history

With { interactive: true } the kernel runs each evaluation through its full main loop, populating In[n], Out[n], and %/%% exactly as in a Mathematica notebook.

const session = new WstpSession(KERNEL, { interactive: true });

// Out[n] is populated automatically; cellIndex reflects the kernel's actual $Line
const r1 = await session.evaluate('Prime[10]');
console.log(r1.cellIndex);      // e.g. 1
console.log(r1.outputName);     // "Out[1]="
console.log(r1.result.value);   // 29

// % and %% return last / second-to-last outputs
const r2 = await session.evaluate('42');
const pct = await session.evaluate('%');
console.log(pct.result.value);  // 42

// Out[n] is accessible from later evaluations
const r3 = await session.evaluate('6 * 7');
const arith = await session.evaluate(`Out[${r3.cellIndex}] + 1`);
console.log(arith.result.value);  // 43

// In[n] stores the input; it evaluates to the result of the expression
const r4 = await session.evaluate('2 + 2');
const inVal = await session.sub(`In[${r4.cellIndex}]`);
console.log(inVal.value);  // 4

// Suppressed evaluations (trailing ;) have empty outputName
// but Out[n] is still stored internally by the kernel
const r5 = await session.evaluate('77;');
console.log(r5.outputName);   // ""
console.log(r5.result.value); // "System`Null"
const out5 = await session.sub(`Out[${r5.cellIndex}]`);
console.log(out5.value);       // 77  (stored internally)

// $Line tracks the kernel counter
const line = await session.sub('$Line');
console.log(line.value);  // r5.cellIndex + 1

session.close();

Streaming output

Callbacks fire in real time as the kernel produces output, before the Promise resolves.

const lines = [];
const r = await session.evaluate(
    'Do[Print["step " <> ToString[k]]; Pause[0.5], {k, 1, 5}]',
    {
        onPrint:   (line) => { lines.push(line); console.log('[live]', line); },
        onMessage: (msg)  => console.warn('[msg]', msg),
    }
);
// lines === ['step 1', 'step 2', 'step 3', 'step 4', 'step 5']
// r.print === ['step 1', 'step 2', 'step 3', 'step 4', 'step 5']  (same data)
// r.result.value === 'Null'

// Use a promise latch if you need to confirm delivery before acting:
let resolveAll;
const allFired = new Promise(r => resolveAll = r);
let count = 0;
await session.evaluate('Do[Print[i]; Pause[0.2], {i, 4}]', {
    onPrint: () => { if (++count === 4) resolveAll(); }
});
await Promise.race([allFired, timeout(5000)]);
console.assert(count === 4);

Concurrent evaluations

All queued evaluations run in strict FIFO order — the link is never corrupted.

// Fire all three at once; results arrive in the same order they were queued.
const [r1, r2, r3] = await Promise.all([
    session.evaluate('Pause[1]; "first"'),
    session.evaluate('Pause[1]; "second"'),
    session.evaluate('Pause[1]; "third"'),
]);
// Total time: ~3 s (serialised, not parallel)
// r1.result.value === 'first', r2.result.value === 'second', etc.

Priority sub() calls

sub() always jumps ahead of queued evaluate() calls — ideal for UI queries like "what is the current value of this variable?" while a long computation is running.

// Start a slow batch job
const batch = session.evaluate('Pause[5]; result = 42');

// While it runs, query progress via sub() — fires after the in-flight eval finishes
// but before any other queued evaluate():
const val = await session.sub('$Version');         // runs next
const pid  = await session.sub('$ProcessID');      // runs after val

await batch;

Abort a long computation

// Use Do[Pause[...]] so the kernel checks for abort signals regularly
const p = session.evaluate('Do[Pause[0.1], {1000}]');

await new Promise(r => setTimeout(r, 500));
session.abort();

const r = await p;
console.log(r.aborted);       // true
console.log(r.result.value);  // '$Aborted'

// Session is still alive — keep evaluating
const r2 = await session.evaluate('2 + 2');
console.log(r2.result.value);  // 4

Dialog subsessions

Dialog[] opens an interactive subsession inside the kernel. Use dialogEval() to send expressions to it and exitDialog() to close it.

// Basic dialog round-trip
const evalDone = session.evaluate('Dialog[]; "finished"', {
    onDialogBegin: (level) => console.log('dialog opened at level', level),
    onDialogEnd:   (level) => console.log('dialog closed at level', level),
});

// Wait for the dialog to open (isDialogOpen flips to true when BEGINDLGPKT arrives)
await pollUntil(() => session.isDialogOpen);

const two = await session.dialogEval('1 + 1');   // { type: 'integer', value: 2 }
const pi  = await session.dialogEval('N[Pi]');   // { type: 'real', value: 3.14159... }

await session.exitDialog();     // sends EnterTextPacket["Return[]"]
const r = await evalDone;       // r.result.value === 'finished'

// exitDialog with a return value
const p2 = session.evaluate('x = Dialog[]; x^2');
await pollUntil(() => session.isDialogOpen);
await session.exitDialog('7');  // Dialog[] returns 7
const r2 = await p2;            // r2.result.value === 49

// dialogEval with Print[] inside
const prints = [];
const p3 = session.evaluate('Dialog[]', {
    onDialogPrint: (line) => prints.push(line),
});
await pollUntil(() => session.isDialogOpen);
await session.dialogEval('Print["hello from the dialog"]');
// Use a promise latch if you need delivery confirmation before exitDialog:
await session.exitDialog();
await p3;
// prints === ['hello from the dialog']

dialogEval('Return[]') does NOT close the dialog. Return[] via EvaluatePacket is unevaluated at top level — there is no enclosing structure to return from. Only exitDialog() (which uses EnterTextPacket) truly exits the dialog.

Variable monitor — peeking at a running loop

You can inspect the current value of any variable while a long computation runs, without aborting it. The trick is to use the Dialog[]/interrupt mechanism to pause the kernel briefly, peek, then resume — all transparent to the running evaluation.

How it works

1. session.interrupt() posts WSInterruptMessage to the kernel.
2. The kernel suspends the running evaluation and sends MENUPKT (the interrupt menu).
3. The C++ backend automatically handles MENUPKT by reading its type+prompt payload and responding with the bare string "i" (inspect mode) — no Wolfram-side handler needed.
4. The kernel opens Dialog[] inline: sends TEXTPKT (option list) → BEGINDLGPKT → INPUTNAMEPKT. isDialogOpen flips to true when INPUTNAMEPKT arrives.
5. The JS monitor calls session.dialogEval('i') to read the current loop variable, then session.exitDialog() to resume.
6. When the main eval resolves the monitor stops.

Wolfram-side interrupt handler (optional): If Internal\AddHandler["Interrupt", Function[{}, Dialog[]]]is installed ininit.wl, the Wolfram handler opens Dialog[]directly and the kernel sendsBEGINDLGPKTwithout going throughMENUPKT. Either path works — the C++ backend handles both the MENUPKT(interrupt-menu) path and the directBEGINDLGPKT` path.

Prerequisites

No special Wolfram-side configuration is required. The C++ backend handles MENUPKT automatically. The evaluated expression must be started with onDialogBegin / onDialogEnd callbacks so the drain loop is in dialog-aware mode:

// The evaluate() call must include dialog callbacks so the drain loop
// handles BEGINDLGPKT / ENDDLGPKT correctly.
const mainEval = session.evaluate(expr, {
    onDialogBegin: (_level) => {},
    onDialogEnd:   (_level) => {},
});

Full example

const { WstpSession } = require('./build/Release/wstp.node');
const KERNEL = '/Applications/Wolfram 3.app/Contents/MacOS/WolframKernel';
const session = new WstpSession(KERNEL);

// Helper: poll until predicate returns true or deadline expires
const pollUntil = (pred, intervalMs = 50, timeoutMs = 3000) =>
    new Promise((resolve, reject) => {
        const deadline = Date.now() + timeoutMs;
        const tick = () => {
            if (pred()) return resolve();
            if (Date.now() > deadline) return reject(new Error('pollUntil: timeout'));
            setTimeout(tick, intervalMs);
        };
        tick();
    });

// Step 1: install the interrupt handler
await session.evaluate(
    'Internal`AddHandler["Interrupt", Function[{}, Dialog[]]]'
);

// Step 2: start the long computation — NOT awaited, runs in background
//         evaluate() accepts onDialogBegin/End so the drain loop services dialogs
const mainEval = session.evaluate(
    'Do[i = k; Pause[0.2], {k, 1, 50}]; "done"',
    {
        onDialogBegin: (_level) => { /* optional: log */ },
        onDialogEnd:   (_level) => { /* optional: log */ },
    }
);

// Step 3: variable monitor — peek at `i` every second until mainEval resolves
let running = true;
mainEval.finally(() => { running = false; });

async function monitor() {
    while (running) {
        await new Promise(r => setTimeout(r, 1000));
        if (!running) break;

        // Send WSInterruptMessage → Wolfram handler opens Dialog[]
        const sent = session.interrupt();
        if (!sent) break;  // session closed or idle

        // Wait for BEGINDLGPKT to arrive (C++ sets isDialogOpen = true)
        try {
            await pollUntil(() => session.isDialogOpen, 50, 3000);
        } catch (_) {
            // Dialog didn't open — computation may have already finished
            break;
        }

        // Read current value of the loop variable
        let val;
        try {
            val = await session.dialogEval('i');
        } catch (e) {
            await session.exitDialog().catch(() => {});
            break;
        }
        console.log(`[monitor] i = ${val.value}`);

        // Resume the main evaluation
        await session.exitDialog();
    }
}

await Promise.all([
    mainEval,
    monitor(),
]);

const r = await mainEval;
console.log('final:', r.result.value);  // "done"
session.close();

Expected output (approximate — timing depends on CPU load):

[monitor] i = 5
[monitor] i = 10
[monitor] i = 15
...
final: done

Extension implementation note

For a VS Code notebook extension using this backend:

• No Wolfram-side interrupt handler is required — the C++ backend handles MENUPKT automatically by responding with "i" (inspect mode). Optionally, installing Internal\AddHandler["Interrupt", Function[{}, Dialog[]]]ininit.wlbypassesMENUPKTentirely and sendsBEGINDLGPKT` directly; both paths are supported.
• The ⌥⇧↵ command flow is: session.interrupt() → poll isDialogOpen → session .dialogEval(cellCode) → render result as "Dialog: Out" in the cell → session .exitDialog().
• The evaluate() call for the main long computation must pass onDialogBegin, onDialogPrint, and onDialogEnd callbacks; these wire the C++ BEGINDLGPKT / ENDDLGPKT handlers that drive isDialogOpen and the dialog inner loop.
• dialogEval() and exitDialog() push to dialogQueue_ in C++, which the drain loop on the thread-pool thread services between kernel packets — no second link/thread needed.
• Dialog results from inspect mode are returned as RETURNTEXTPKT (OutputForm text) rather than RETURNPKT (full WL expression); the C++ SDR layer parses these transparently.

Real-time side channel (WstpReader)

Use WstpReader to receive kernel-pushed data while a long evaluation is running on the main link.

// Step 1: create the push link inside the kernel and get its name
await session.evaluate('$pushLink = LinkCreate[LinkProtocol -> "TCPIP"]');
const { result: nameExpr } = await session.evaluate('$pushLink[[1]]');
const linkName = nameExpr.value;  // e.g. "[email protected],[email protected]"

// Step 2: connect the JS reader
const reader = new WstpReader(linkName, 'TCPIP');

// Step 3: start the kernel writer (NOT awaited — runs concurrently)
const bgWriter = session.evaluate(
    'Do[LinkWrite[$pushLink, {i, RandomReal[]}]; Pause[0.5], {i, 1, 10}];' +
    'Pause[1]; LinkClose[$pushLink]; "writer done"'
);

// Step 4: read 10 values in real time
const received = [];
try {
    for (let i = 0; i < 10; i++) {
        const v = await reader.readNext();
        // v = { type: 'function', head: 'List', args: [{value:i}, {value:rand}] }
        received.push(v);
        console.log(`item ${i + 1}:`, v.args[0].value, v.args[1].value);
    }
} finally {
    reader.close();
    try { await bgWriter; } catch (_) {}
}

Parallel independent kernels

Each WstpSession is an entirely separate process with its own state.

// Launch two kernels in parallel
const [ka, kb] = await Promise.all([
    Promise.resolve(new WstpSession(KERNEL)),
    Promise.resolve(new WstpSession(KERNEL)),
]);

// Run independent computations simultaneously
const [ra, rb] = await Promise.all([
    ka.evaluate('Sum[1/k^2, {k, 1, 10000}]'),
    kb.evaluate('Sum[1/k^3, {k, 1, 10000}]'),
]);

console.log(ra.result);  // Pi^2/6 approximation
console.log(rb.result);  // Apéry's constant approximation

ka.close();
kb.close();

Error Handling

| Situation | Behaviour | |-----------|-----------| | Syntax error in expr | Kernel sends a message; evaluate() resolves with messages: ['...'] and result: { type: 'symbol', value: 'Null' } or '$Failed' | | Expression too deep (> 512 nesting levels) | evaluate() rejects with "expression too deep" — the session stays alive | | Abort | evaluate() resolves with aborted: true, result.value === '$Aborted' | | Kernel crashes | evaluate() rejects with a link error message — create a new WstpSession | | dialogEval() / exitDialog() when no dialog open | Rejects with "no dialog subsession is open" | | dialogEval() / exitDialog() when flushed by closeAllDialogs() | Rejects with "dialog closed by closeAllDialogs" | | abort() / closeAllDialogs() flushes dialog queue | Pending dialogEval()/exitDialog() promises reject immediately | | evaluate() after close() | Rejects with "Session is closed" | | WstpReader.readNext() after link closes | Rejects with a link-closed error |

// Robust evaluate wrapper
async function safeEval(session, expr) {
    try {
        const r = await session.evaluate(expr);
        if (r.aborted)           return { ok: false, reason: 'aborted' };
        if (r.messages.length)   console.warn('kernel messages:', r.messages);
        return { ok: true, result: r.result };
    } catch (e) {
        return { ok: false, reason: e.message };
    }
}

Diagnostic Logging

Two mechanisms — both disabled by default, zero overhead when off:

setDiagHandler(fn) — JS callback

const { setDiagHandler } = require('./build/Release/wstp.node');

setDiagHandler((msg) => {
    process.stderr.write(`[${new Date().toISOString().slice(11, 23)}] ${msg}\n`);
});

// Measure TSFN delivery latency:
//   C++ logs: "[TSFN][onPrint] dispatch +142ms ..."
//   Your handler timestamp - 142ms = module load time offset
//   Comparing both gives end-to-end callback delivery time

setDiagHandler(null);  // disable

DEBUG_WSTP=1 — direct stderr from C++

DEBUG_WSTP=1 node compute.js 2>diag.txt
cat diag.txt
# [wstp +23ms] [Session] restart attempt 1 — $Output routing broken on previous kernel
# [wstp +1240ms] [WarmUp] $Output routing verified on attempt 1
# [wstp +1503ms] [Eval] pkt=8
# [wstp +1503ms] [Eval] pkt=2
# [wstp +1503ms] [TSFN][onPrint] dispatch +1503ms "step 1"
# ...

Timestamps are module-relative milliseconds (since the addon was loaded).