WASM Runner

For managing and executing WASM on browser, mobile and NodeJS applications.

Here's a Demo App for reference.

Getting Started

Prerequisites

Here are the tools that you need for your setup.

• NodeJS (we recommend to use v20 or later) for running and developing this project.

This package is published under two scopes:

• @coherentglobal/wasm-runner — public, on the npm registry (npmjs.org). No auth needed.
• @coherentcapital/wasm-runner — the same build, published to the public npm registry and to GitHub Packages (private, under the CoherentCapital org).

Install

The public package needs no extra setup:

npm install "@coherentglobal/wasm-runner"
# or
yarn add "@coherentglobal/wasm-runner"

To install the @coherentcapital/wasm-runner build from GitHub Packages, point the scope at the GitHub registry and authenticate with a personal access token that has the read:packages scope. Add this to your project (or user) .npmrc:

@coherentcapital:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}

then:

npm install "@coherentcapital/wasm-runner"

Usage

Browser ( HTML )

Runner can be used in an HTML by adding the minified JS file to your project.

<script src="https://wasm-runner-sdk.s3.ap-southeast-1.amazonaws.com/wasmrunner.min.js"></script>

Have your javascript initialize the model and run some values through the execute function.

/**
 * `id` defines the model version ID and the `url` is
 * the path of the zip relative to this file.
 */
const modelConfig = {
  id: "f30f5935-36c2-4155-aede-523ab2245fd6",
  url: "<zip url>",
};

/**
 * Input values to be executed on the model. The structure and sample
 * data can be found on Spark API Tester.
 */
const payload = {
  request_data: {
    inputs: {
      age: 60,
      insure: "Insure",
      medical: "Yes",
      plan: "Plan 1",
    },
  },
  request_meta: {
    service_uri: "",
    service_uuid: "",
    version: "",
    version_uuid: "f30f5935-36c2-4155-aede-523ab2245fd6",
    transaction_date: "2021-08-18T04:17:17.142Z",
    call_purpose: "postman_request",
    source_system: "",
    correlation_id: "",
    requested_output: "",
  },
};

const wasmRunner = new WasmRunner(modelConfig);
await wasmRunner.initialize();
const response = await wasmRunner
  .execute(payload)
  .catch((err) => console.log(err));

// Do something with the response

ReactJS

You may follow the normal installation follow then import the packge into your project.

import { WasmRunner } from "@coherentglobal/wasm-runner";

Then do model initialize and execute.

/**
 * `id` defines the model version ID and the `url` is
 * the path of the zip relative to this file.
 */
const modelConfig = {
  id: "f30f5935-36c2-4155-aede-523ab2245fd6",
  url: "<zip url>",
};

/**
 * Input values to be executed on the model. The structure and sample
 * data can be found on Spark API Tester.
 */
const payload = {
  request_data: {
    inputs: {
      age: 60,
      insure: "Insure",
      medical: "Yes",
      plan: "Plan 1",
    },
  },
  request_meta: {
    service_uri: "",
    service_uuid: "",
    version: "",
    version_uuid: "f30f5935-36c2-4155-aede-523ab2245fd6",
    transaction_date: "2021-08-18T04:17:17.142Z",
    call_purpose: "postman_request",
    source_system: "",
    correlation_id: "",
    requested_output: "",
  },
};

const wasmRunner = new WasmRunner(modelConfig);
await wasmRunner.initialize();
const response = await wasmRunner
  .execute(payload)
  .catch((err) => console.log(err));

// Do something the response

NodeJS

You may follow the normal installation flow then import the packge into your project.

const { WasmRunner } = require("@coherentglobal/wasm-runner");

/**
 * `id` defines the model version ID and the `url` is
 * the path of the zip relative to this file.
 */
const modelConfig = {
  id: "f30f5935-36c2-4155-aede-523ab2245fd6",
  url: "<zip url>",
};

/**
 * Input values to be executed on the model. The structure and sample
 * data can be found on Spark API Tester.
 */
const payload = {
  request_data: {
    inputs: {
      age: 60,
      insure: "Insure",
      medical: "Yes",
      plan: "Plan 1",
    },
  },
  request_meta: {
    service_uri: "",
    service_uuid: "",
    version: "",
    version_uuid: "f30f5935-36c2-4155-aede-523ab2245fd6",
    transaction_date: "2021-08-18T04:17:17.142Z",
    call_purpose: "postman_request",
    source_system: "",
    correlation_id: "",
    requested_output: "",
  },
};

const wasmRunner = new WasmRunner(modelConfig);
await wasmRunner.initialize();
const response = await wasmRunner
  .execute(payload)
  .catch((err) => console.log(err));

Ways to load and initialize model

WASM Runner parameter

You may pass the cofig parameters directly to WasmRunner class whenever you create an instance.

const wasmRunner = new WasmRunner({
  id: "f30f5935-36c2-4155-aede-523ab2245fd6",
  url: "<zip url>",
});

// Trigger initialize to load the passed model
await wasmRunner.initialize();

Append

Pass the model config to append function after instantiation.

const wasmRunner = new WasmRunner();
// Pass the config details to `append` function
await wasmRunner.append({
  id: "f30f5935-36c2-4155-aede-523ab2245fd6",
  url: "<zip url>",
});

Combination of both

For scenarios that you need to load a lot of models, you may do both. During the instance creation, you may pass a list of configs to WasmRunner then do append for additional models.

const initialModels = [
  {
    id: "f30f5935-36c2-4155-aede-523ab2245fd6",
    url: "<zip url>",
  },
  {
    id: "a5gf5935-36c2-6621-aede-a41ab226aa53",
    url: "<zip url>",
  },
];

const additionalModel = {
  id: "e8ba9e7e-169c-4752-8a8e-d6eb0c78cb01",
  url: "<zip url>",
};

const wasmRunner = new WasmRunner(initialModels);
await wasmRunner.initialize();
await wasmRunner.append(additionalModel);

const response = await wasmRunner
  .execute(payload)
  .catch((err) => console.log(err));

Development

Setup

Clone the repo, then install the dependencies

$ git clone https://github.com/CoherentCapital/wasm-runner-js.git
$ yarn install

Test

yarn test

Build

For generating distribution package

yarn build

Functions

initialize()

Triggers the loading and integration of WASM files to Runner module. This step is required before the execute step.

Returns: Promise<void>

append(modelConfig)

Could be an alternative to initialize(). This loads additional model that you might need during execute after the Runner instantiation.

Returns: Promise<void>

execute(payload)

Performs the model calculation. Most common approach, Runner will use request_meta.version_id from the payload to locate the model to execute. So it is necessary to ensure that the model is loaded, could be through Runner instantiation or via append.

Returns: Promise<Object>

isExist(id)

Checks if model exists from the list of initialized models.

Returns: Promise<Boolean>

remove(id)

Removes a model. This is the way to clean up memory allocation for unused loaded models.

Returns: Promise<void>

dispose()

Disposes all resources: terminates all running models and cleans up temporary files. Use this method when you're done with the WasmRunner instance to ensure proper cleanup.

Returns: Promise<void>

const wasmRunner = new WasmRunner(modelConfig);
await wasmRunner.initialize();

// ... perform operations ...

// Clean up when done
await wasmRunner.dispose();

Columnar Serializer

ColumnarSerializer converts between row format (an array of plain objects, as a caller would naturally write it) and columnar format (a header row followed by value rows). The columnar form deduplicates repeated keys, which is why it is used for table-shaped payloads crossing into the WASM model and out to XCall / XConnector resolvers.

import { ColumnarSerializer } from "@coherentglobal/wasm-runner";

const serializer = new ColumnarSerializer();

serializer.serialize([{ a: 1 }, { a: 2 }]); // => [["a"], [1], [2]]
serializer.deserialize([["a"], [1], [2]]); // => [{ a: 1 }, { a: 2 }]

Every example below has been executed against the implementation. Where behavior is surprising, that is stated plainly rather than smoothed over.

Serializer API

| Member | Kind | Notes | | ------------------------------------------- | -------- | ----------------------------------------------------------------------------------------------------- | | serialize(data) | instance | Row → columnar. Returns data unchanged if it already looks columnar. | | deserialize(data) | instance | Columnar → row. Returns data unchanged if it does not look columnar. Do not pass a JSON string. | | serializeString(json) | instance | String in → string out. Use this for JSON strings. | | deserializeString(json) | instance | String in → string out. Use this for JSON strings. | | ColumnarSerializer.isDeserializable(data) | static | Is this payload in columnar form? Accepts an object or a JSON string. | | ColumnarSerializer.isEmpty(data) | static | No data in any row: all-blank columnar forms, plus bare all-null ([null]). Accepts a JSON string. |

Columnar format

A columnar payload is an array of rows:

[ headerRow, dataRow, dataRow, ... ]
   index 0    index 1   index 2

• Index 0 is the header row. Always a non-null array. Header entries must be strings or numbers; anything else throws Column header must be a string or number.
• Index >= 1 is a data row. Either an array of values positionally aligned to the header row, or a literal null meaning a blank row.
• Positions shift by one. Columnar index i holds data row i - 1. So in [["key"], null, ["value"]] the null is the first data row: [null, { key: "value" }].

Header derivation

Headers are the union of all keys across all rows, in first-seen order. Values are placed at their header index, so rows with different key sets stay aligned:

serializer.serialize([
  { a: 1, b: 2 },
  { b: 3, a: 4 },
]); // => [["a","b"], [1,2], [4,3]]
serializer.serialize([{ b: 1 }, { a: 2 }]); // => [["b","a"], [1], [null,2]]

The second case is a ragged row: { b: 1 } has no a, so index 0 of its value row is an array hole. In memory that reads as undefined; over JSON it becomes null. This is why ragged rows do not survive a round trip — see "Ragged rows gain explicit null properties" under Serializer quirks.

Null rows

A null row means a blank row and round-trips as null:

serializer.serialize([{ a: 1 }, null, { a: 2 }]); // => [["a"], [1], null, [2]]
serializer.deserialize([["a"], [1], null, [2]]); // => [{ a: 1 }, null, { a: 2 }]

null is only valid at index >= 1. A null at index 0 would be a null header row, which is meaningless, so such a payload is not columnar and passes through deserialize untouched.

null appears at two different depths, and only one of them is unambiguous:

[["a","b"], [1], null, [null, 2]]
                 ^^^^   ^^^^
              blank row  null cell

A row-level null is unambiguous — it can only mean a blank row, because a row that merely contains a null is one level deeper:

serializer.serialize([{ a: 1 }, null]); // => [["a"], [1], null]
serializer.serialize([{ a: 1 }, { a: null }]); // => [["a"], [1], [null]]

A cell-level null is not. It means either an explicitly null value or a key the row never had, and the two are indistinguishable:

serializer.serializeString('[{"a":1},{"b":2}]'); // => '[["a","b"],[1],[null,2]]'
serializer.serializeString('[{"a":1},{"a":null,"b":2}]'); // => '[["a","b"],[1],[null,2]]'

Both read back as { a: null, b: 2 } — see "Ragged rows gain explicit null properties" under Serializer quirks.

Keyless payloads

When there are no keys anywhere there is nothing for the header row to describe. If at least one row still had to be encoded ({} cannot appear as itself in columnar form, so it becomes []), an empty header row is prepended:

serializer.serialize([{}]); // => [[], []]
serializer.serialize([{}, null]); // => [[], [], null]

The header row must be prepended, not appended — rows are positional, so appending would encode [{}, null] as [[], null, []], which reads back with the rows swapped.

When every row is null, nothing needs encoding, so the nulls pass through bare with no header row:

serializer.serialize([null]); // => [null]
serializer.serialize([null, null]); // => [null, null]

These outputs are deliberately not columnar (null at index 0): they join [] and all-scalar arrays in the pass-through family, and deserialize returns them unchanged rather than decoding them. As a nested cell value, bare nulls survive via the array-mapping branch, so [{ t: [null] }] still round-trips (as [["t"], [[null]]]). One consequence to be aware of: a consumer that expects serialize output to always be columnar will see a plain array instead. isEmpty recognizes the bare all-null family directly — isEmpty(serializer.serialize([null])) is true — which means isEmpty(x) no longer implies isDeserializable(x); it implies deserialize(x) carries no data. The previous encodings [[], null] and [[], null, null] remain valid input and still decode to [null] / [null, null].

Nesting

An object or array value is serialized recursively. A cell holding an array of objects is itself a columnar payload:

serializer.serialize([{ t: [{ x: 1 }, null, { x: 2 }] }]);
// => [["t"], [[["x"], [1], null, [2]]]]

Format invariants

1. Index 0 is always a non-null array.
2. null at index >= 1 is exactly one blank data row at that position.
3. null is never a header row and never a header slot.
4. Cell-level null means an explicit null value or a missing cell in a ragged row.
5. serialize is idempotent: columnar output is recognized and returned unchanged; pass-through output (see 6) is stable under repeated application.
6. Every payload serialize emits either satisfies isDeserializable or is a pass-through form that deserialize returns unchanged ([], all-scalar arrays, and all-null arrays like [null]) — with one genuinely broken exception, see [undefined] below.

Round-trip guarantees

deserialize(serialize(x)) returns x for:

• Arrays of plain objects whose rows share the same key set
• Arrays containing null rows, in any position from index 1 onward
• Keyless shapes: [], [{}], [{}, null] — plus [null] and [null, null], which round-trip via pass-through (their serialized form is not columnar; see "Keyless payloads")
• Nested arrays of objects (1D tables) as cell values, at arbitrary depth, with null rows at any level — including all-null cells like [{ t: [null] }]

It does not round-trip for the cases below. These are limitations of the format, not regressions; each is pinned by a test in tests/parser/columnar/columnarSerializer.test.ts.

Serializer quirks

Ragged rows gain explicit null properties

Rows are read back against the union header row (see "Header derivation" above), and the two halves of a ragged payload behave differently:

serializer.serialize([{ a: 1 }, null, { b: 2 }]);
// => [["a","b"], [1], null, [null, 2]]   (in memory the null cell is an array hole)

serializer.deserialize([["a", "b"], [1], null, [null, 2]]);
// => [{ a: 1 }, null, { a: null, b: 2 }]

The [1] row is simply short, so its missing b produces no property at all — short rows lose nothing. But the [null, 2] row carries an explicit cell null, which becomes a present a: null property — wire-transported holes gain explicit nulls. So { b: 2 } comes back as { a: null, b: 2 }: not a round trip. (In memory, before JSON, the gained property is present-but-undefined, which JSON.stringify hides.) The null row itself survives exactly. If absent-vs-null matters, strip null-valued keys after deserializing.

serialize cannot serialize genuine 2D data

serialize decides "already columnar?" by shape alone — an array whose rows are all arrays. So any real 2D array is assumed to be an already-serialized payload and passes through untouched, then deserializes as if its first row were headers:

serializer.serialize([
  [1, 2],
  [3, 4],
]); // => [[1,2],[3,4]]   (unchanged — treated as columnar)
serializer.deserialize([
  [1, 2],
  [3, 4],
]); // => [{ "1": 3, "2": 4 }]   (numbers became headers)

Numeric headers are legal, which is what makes this silent.

A 2D array inside a cell loses all but its first row

The single-object encoding [[headers, ...rows]] is shape-identical to a cell holding a one-row 2D table, so the detection heuristic misreads it and only the first row survives:

serializer.deserialize(serializer.serialize({ Table: [[{ x: 1 }, { x: 2 }]] }));
// => [{ Table: { x: 1 } }]   -- {x:2} is gone
serializer.deserialize(serializer.serialize([[{ a: 1 }, null]])); // => []

This is null-independent — the first example has no null in it. Fixing it needs a format-level way to distinguish the two shapes, which would be a wire-format change. Use 1D tables (arrays of objects) as cell values; they round-trip correctly.

Memoization returns stale results for mutated inputs

All four internal helpers are memoized with a 100-entry cache keyed on argument identity. Mutating an array after serializing it returns the previous result:

const rows = [{ x: 1 }];
serializer.serialize(rows); // => [["x"], [1]]
rows.push({ x: 2 });
serializer.serialize(rows); // => [["x"], [1]]        <-- stale
// ...100+ other calls later, the entry is evicted:
serializer.serialize(rows); // => [["x"], [1], [2]]   <-- now correct

The staleness window therefore depends on cache pressure. Treat inputs as immutable: build a new array rather than mutating one you have already passed in.

deserialize silently corrupts JSON strings

isDeserializable accepts a JSON string (it parses it to inspect the shape), but deserialize then hands the raw string to the transform, which iterates it character by character:

serializer.deserialize('[["a"],[1]]');
// => [{"[":"["},{"[":"\""},{"[":"a"}, ...]   garbage

Use deserializeString / serializeString for strings.

A top-level null throws

Blank rows are handled, but a bare null payload still reaches Object.keys(null):

serializer.serialize(null); // TypeError: Cannot convert undefined or null to object

[undefined] produces an invalid payload

An undefined row takes the scalar branch and emits a payload whose header row is undefined, which JSON-encodes to null — a null header row, which is by definition not columnar, so it will not deserialize:

serializer.serialize([undefined]); // => [undefined]  ->  JSON: [null]

undefined values are fine — they become null, and deserialize as null.

Scalars and class instances are mangled, not rejected

Only own enumerable properties survive. Anything whose data lives elsewhere is emptied, and scalars are treated as key-value bags:

serializer.serialize(5); // => [[], []]
serializer.serialize("hi"); // => [["0","1"], ["h","i"]]
serializer.serialize([{ d: new Date(0) }]); // => [["d"], [[[], []]]]   Date is destroyed

Dates, Map, Set, and class instances must be converted to plain data first.

A non-array input always comes back wrapped in an array

serialize of a plain object emits [headers, values], which deserializes to a one-element array. Nested plain objects are affected the same way:

serializer.deserialize(serializer.serialize({ a: 1, b: 2 })); // => [{ a: 1, b: 2 }]  (was an object)
serializer.deserialize(serializer.serialize([{ a: { b: 1 } }])); // => [{ a: [{ b: 1 }] }]  (was an object)

The object-vs-array distinction is not preserved.

Mixed scalar and object rows do not round-trip

An array holding both scalars and objects takes a special branch that emits a shape nothing reads back:

serializer.serialize([1, { a: 2 }]); // => [1, [[["a"], [2]]]]
serializer.serialize([{ a: 1 }, 2]); // => [[1], [[["a"], 2]]]
serializer.serialize(["x", "y"]); // => ["x","y"]   (all scalars: unchanged)

Deserialize errors

| Condition | Message | | --------------------------------------------------------------- | ------------------------------------------------------------------------ | | More values in a row than headers | Error: Empty column detected at row {i} and column {j} | | Header entry is not a string or number | Error: Column header must be a string or number. Found {type} instead. | | Header row is null (reachable only via a crafted nested cell) | Error: Header row cannot be null |

Behavior changes

Everything below changed relative to 0.6.0. Each row was produced by running both implementations side by side. Payloads with no null anywhere are byte-identical in both directions — only the cases here move.

serialize

| Input | 0.6.0 | Now | | ---------------------- | -------------------------- | ---------------------------- | | [{a:1}, null, {a:2}] | [["a"],[1],[],[2]] | [["a"],[1],null,[2]] | | [null] | [[],[]] | [null] | | [null, null] | [[],[],[]] | [null,null] | | [{}, null] | [[],[],[]] | [[],[],null] | | [{t:[{x:1}, null]}] | [["t"],[[["x"],[1],[]]]] | [["t"],[[["x"],[1],null]]] | | [{}], [] | unchanged | unchanged |

A blank row is no longer flattened into an empty row, so null and {} are now distinguishable. An all-null array loses its synthetic header row (see "Keyless payloads"); [{}, null] keeps one, because {} still has to be encoded.

deserialize

| Input | 0.6.0 | Now | | ---------------------- | --------------------------------- | ---------------------- | | [["a"],[1],null,[2]] | returned unchanged (not columnar) | [{a:1}, null, {a:2}] | | [[], null] | returned unchanged (not columnar) | [null] | | [["a"],[1],[],[2]] | [{a:1}, {}, {a:2}] | unchanged | | [null] | returned unchanged | unchanged |

The first two are the substantive change: a payload containing a null row used to fail format detection and pass straight through undecoded. The empty-row encoding still decodes to {}, so payloads produced by 0.6.0 — and by any consumer still coercing blank rows to [] — read back exactly as before.

Round-tripping [{a:1}, null, {a:2}] therefore returns [{a:1}, null, {a:2}] where it previously returned [{a:1}, {}, {a:2}].

Predicates

| Call | 0.6.0 | Now | | --------------------------------- | ------------------------------------- | --------- | | isDeserializable([["a"], null]) | false | true | | isEmpty([null]) | false | true | | isEmpty([[], null]) | false | true | | isEmpty("[[],[]]") | throws data.every is not a function | true | | isDeserializable([null]) | false | unchanged |

isDeserializable is a strict superset — it accepts null data rows and still rejects a null header row, so nothing that was columnar stopped being columnar. isEmpty gained the bare all-null family and now parses strings instead of throwing.

String variants

serializeString and deserializeString inherit all of the above; e.g. deserializeString('[["a"],[1],null,[2]]') returned its input verbatim in 0.6.0 and now returns '[{"a":1},null,{"a":2}]'.

Wire-format compatibility

Null-row preservation widens the format: a consumer that requires every row to be an array must relax that check before it can receive null rows. The relaxed check is a superset — every previously valid payload stays valid — so no existing payload changes meaning.

Two other observable changes for consumers:

• An all-null input now serializes to a bare non-columnar array ([null], see "Keyless payloads"), so code expecting every serialize output to be columnar will receive plain data for that shape.
• isEmpty returns true for the bare all-null family ([null], [null, null]) and accepts JSON strings instead of throwing. Both are additive — no payload that was previously true became false — but code relying on "isEmpty(x) implies x is columnar" must use isDeserializable(x) for that check instead.

Known consumers

ColumnarSerializer is consumed by wasm-server (src/utils.js, src/logic/runnerLogic.js) and wasm-server-lambda (batchProcessingHandler/, lib/). Both import it from this package rather than reimplementing the format, so there is no second JavaScript predicate to keep in sync — upgrading the dependency moves both sides at once. Both pin ^0.6.0, which does not resolve to 0.7.0, so neither picks this up until its dependency is bumped explicitly.

Points confirmed against those repos:

• Neither calls ColumnarSerializer.isEmpty. Every isEmpty in both is a local object helper, so the implication change above has no consumer today.
• wasm-server never emits null rows. toColumnarRows coerces them to [] before serializing (SP-16416), for the same Object.keys(null) crash this release fixes. That coercion is now unnecessary but remains harmless, and it means server → client responses keep sending [], which deserializes to {} as before. Null preservation is therefore client → server only until that coercion is removed.
• No branch depends on a null row flipping isDeserializable to false. The batch-merge in runnerLogic.js silently drops rows when two chunks classify differently; today a chunk containing a null row classifies as non-columnar while one without it classifies as columnar, which is exactly that mismatch. The superset predicate makes both classify as columnar, closing that path rather than opening one.

The ExcelEngine (.NET) parses these payloads independently of this package and was not audited here — it is the one consumer whose handling of a null row is unverified. Since wasm-server pre-coerces blank rows to [], it may never receive one today.

Limitations

Lower Level Error

Please note that if there's lower level error that occurs, wasm runner can't handle that kind of exception.