Spark Execute SDK

Execute Spark models both online and offline with a unified API. This SDK seamlessly switches between local WebAssembly execution and remote API calls, giving you the flexibility to run models wherever makes sense for your application.

Features

• Run models locally using WebAssembly (offline)
• Call Spark API endpoints (online)
• Automatic fallback from offline to online on errors
• Cancel long-running executions mid-flight
• Automatic memory management with configurable limits
• Full support for cross-service calls (XCall)
• Works in Node.js and browser environments

Quick Start

Install the package:

npm install @coherentglobal/spark-execute-sdk

Basic usage:

// CommonJS
const Spark = require('@coherentglobal/spark-execute-sdk');

// ES Modules / TypeScript
import Spark from '@coherentglobal/spark-execute-sdk';

const spark = new Spark({
  sparkEndpoint: {
    url: "https://excel.uat.us.coherent.global",
    tenant: "your-tenant",
    authType: "public"
  }
});

const result = await spark.execute({
  request_data: { inputs: { Input: 1 } },
  request_meta: { version_id: "model-uuid" }
});

await spark.destroy();

How-to Guides

Installation

npm install @coherentglobal/spark-execute-sdk

or

yarn add @coherentglobal/spark-execute-sdk

Usage

Configuration

Execution Modes:

• Include both sparkEndpoint and nodegenModels for automatic fallback (offline → online)
• Include only sparkEndpoint for online-only execution
• Include only nodegenModels for offline-only execution

const config = {
  // Online execution (optional - omit for offline-only)
  sparkEndpoint: {
    url: "https://excel.uat.us.coherent.global",
    tenant: "tenant",
    authType: "public" | "syntheticKey" | "bearerToken",

    // Provide value OR function (called per request - you handle caching)
    syntheticKey: "apiKey" || generateSyntheticKey(),
    bearerToken: "token" || generateToken(),
  },

  // Offline execution (optional - omit or use [] for online-only)
  nodegenModels: [
    {
      versionId: "uuid",
      type: "base64", // Always "base64"
      binary: string | Blob | Buffer | ArrayBuffer | Function, // See below
      preventCleanup: false, // Keep in memory even when limit reached
      replica: 1, // Number of concurrent worker instances
      metaData: {
        EngineInformation: {
          ProductName: "folder-name",
          ServiceName: "service-name",
          VersionId: "uuid"
        }
      }
    }
  ],

  // Advanced options
  offlineMaxService: 20, // Max models in memory before LRU cleanup
  timeout: 60000 // Request timeout (ms)
};

Understanding the binary Configuration

The binary field accepts different formats depending on your environment.

| Environment | Accepted Formats | |-------------|------------------| | Browser | Base64 string, Blob, ArrayBuffer, Function | | Node.js | Base64 string, Buffer, ArrayBuffer, Function |

Important: The type field should always be "base64" regardless of actual binary format.

Common Patterns

Static values:

// Base64 string (most common)
const fs = require('fs');
binary: fs.readFileSync('./model.zip', 'base64')

// Buffer (Node.js)
binary: fs.readFileSync('./model.zip')

// ArrayBuffer (fetch in browser/Node.js)
const response = await fetch('/models/model.zip');
binary: await response.arrayBuffer()

// Blob (browser file upload)
binary: new Blob([zipData], { type: 'application/zip' })

Dynamic loading (function):

// Lazy load from filesystem
binary: (versionId) => require('fs').readFileSync(`./models/${versionId}.zip`, 'base64')

// Fetch from remote storage
binary: async (versionId) => {
  const response = await fetch(`https://storage.example.com/models/${versionId}.zip`);
  return await response.arrayBuffer();
}

// With caching
const modelCache = new Map();
binary: async (versionId) => {
  if (!modelCache.has(versionId)) {
    modelCache.set(versionId, await fetchModelFromS3(versionId));
  }
  return modelCache.get(versionId);
}

API Reference

Initialization

Create a Spark instance with your configuration:

const spark = new Spark(config);

execute(input, version_id?)

Execute a model using the provided input payload.

// Basic usage
const response = await spark.execute(input);

// With explicit version ID
const response = await spark.execute(input, 'model-uuid-here');

Tip: The version_id parameter is optional if already in request_meta. See Error Handling below.

executeWithCancellationToken(input, version_id?)

Execute a model with the ability to cancel the operation mid-execution. Useful for long-running calculations that users may want to cancel.

const execution = spark.executeWithCancellationToken(input);

// Cancel anytime before completion
execution.cancellationToken.cancel();

execution.response
  .then((result) => {
    console.log('Completed:', result);
  })
  .catch((err) => {
    if (err instanceof Spark.WasmRunnerErrors.ExecuteCancelled) {
      console.log('User cancelled the execution');
    } else {
      console.error('Execution failed:', err);
    }
  });

Important: Cancellation is best-effort. The underlying WebAssembly execution may continue running in its worker thread, but the result will be discarded and an ExecuteCancelled error will be thrown. The worker is cleaned up after completion.

destroy()

Clean up all resources, unload models, and terminate worker threads. Call this when you're done with a Spark instance to prevent memory leaks.

await spark.destroy();

Critical: Always call destroy() when finished, especially in browsers or long-running Node.js apps. Forgetting this can lead to memory leaks as WebAssembly modules and workers won't be garbage collected.

Error Handling

Specific error types accessible via Spark.WasmRunnerErrors:

try {
  const response = await spark.execute(input);
} catch (err) {
  if (err instanceof Spark.WasmRunnerErrors.MissingModelError) {
    console.error('Model not found:', err.message);
  } else if (err instanceof Spark.WasmRunnerErrors.ExecuteCancelled) {
    console.error('Execution was cancelled');
  } else if (err instanceof Spark.WasmRunnerErrors.UnauthorizedError) {
    console.error('Authentication failed');
  } else if (err instanceof Spark.WasmRunnerErrors.BadRequestError) {
    console.error('Bad request:', err.message);
  } else {
    console.error('Unknown error:', err);
  }
}

Available Error Types:

• MissingModelError - Model not found
• ExecuteCancelled - Execution was cancelled via cancellation token
• UnauthorizedError - Authentication failed (invalid token/key)
• BadRequestError - Invalid request or SDK not initialized properly

Error Response Structure

When an online API request fails, the Spark API returns structured error responses:

{
  "error": {
    "status": 401,
    "message": "Unauthorized: Invalid API key",
    "code": "UNAUTHORIZED"
  }
}

Common HTTP status codes:

• 400 Bad Request - Invalid input data or malformed request
• 401 Unauthorized - Missing or invalid authentication credentials
• 404 Not Found - Model or endpoint not found
• 500 Internal Server Error - Server-side error during execution
• 503 Service Unavailable - Service temporarily unavailable

Input Structure

{
  "request_data": { "inputs": { "Input": 1 } },
  "request_meta": {
    "version_id": "<model id>",
    "transaction_date": "2022-09-19T04:17:17.142Z",
    "call_purpose": "Spark - API Tester"
    // correlation_id, source_system, requested_output (optional)
  }
}

Platform-Specific Usage

Node.js

// CommonJS
const Spark = require('@coherentglobal/spark-execute-sdk');

// ES Modules / TypeScript
import Spark from '@coherentglobal/spark-execute-sdk';

const fs = require('fs');
const spark = new Spark({
  nodegenModels: [{
    versionId: "model-uuid",
    binary: fs.readFileSync('./model.zip', 'base64'),
    metaData: { /* ... */ }
  }]
});

const response = await spark.execute(input);
await spark.destroy();

Browser

<script src="node_modules/@coherentglobal/spark-execute-sdk/dist/browser.js"></script>
<script>
  const config = {
    nodegenModels: [{
      versionId: "model-uuid",
      binary: base64EncodedModel,
      metaData: { /* ... */ }
    }]
  };

  const spark = new Spark(config);

  spark.execute(input)
    .then(response => console.log('Result:', response))
    .catch(err => console.error('Error:', err))
    .finally(() => spark.destroy()); // Always cleanup!
</script>

Troubleshooting

Common Issues

"Model not found" Error

Problem: SDK throws MissingModelError when trying to execute.

Solutions:

• Verify that versionId in your input matches a configured model in nodegenModels
• Check that request_meta.version_id is correctly specified in your input
• Check if the model binary is loaded (check browser console for loading errors)

"Binary corrupted" or Initialization Errors

Problem: Model fails to initialize or throws corruption errors.

Solutions:

• Verify the model binary is valid and not truncated during download
• Check that the base64 encoding is correct (no line breaks or extra characters)
• Try re-downloading the model from Spark
• Ensure the binary isn't corrupted (use type: "base64" for base64-encoded zips)

"Worker terminated unexpectedly"

Problem: WebAssembly worker crashes during execution.

Solutions:

• Check browser console for memory errors
• Reduce the number of concurrent replicas
• Ensure the model isn't too large for the available memory
• Try increasing timeout value if the model is computationally intensive

ExecuteCancelled Thrown Instantly

Problem: Cancellation error occurs immediately even without calling cancel.

Solutions:

• Check if cancellation token is being reused from a previous execution
• Create a new execution instance for each call to executeWithCancellationToken
• Verify that cancel() isn't being called earlier in your code flow

Online Fallback Not Triggered

Problem: SDK doesn't fall back to online execution when offline fails.

Solutions:

• Ensure both sparkEndpoint and nodegenModels are configured
• Check that the offline error is not being caught before fallback occurs
• Verify network connectivity for online execution
• Check authentication credentials for the online endpoint

Memory Leaks in Long-Running Applications

Problem: Application memory usage keeps increasing over time.

Solutions:

• Always call spark.destroy() when finished with an instance
• Don't create multiple Spark instances unnecessarily
• Monitor the number of loaded models (check against offlineMaxService limit)
• Use preventCleanup: false for models that don't need to persist

Memory Usage & Cleanup

How Model Cleanup Works

The SDK automatically manages memory using LRU (Least Recently Used) eviction:

• offlineMaxService (default: 20): Maximum models kept in memory
• preventCleanup: Set to true to prevent a model from being unloaded
• When the limit is reached, the least recently used model (without preventCleanup) is unloaded

Worker Thread Management

Each model uses dedicated worker threads for parallel execution.

Concurrency Model

• Each model creates a pool of isolated worker threads based on the replica count (default: 1)
• Incoming execution requests are distributed across available workers in round-robin fashion
• Each worker runs in its own thread with its own WebAssembly instance—no shared state
• Model are lazy-loaded on first run
• Higher replica counts increase initial execution time (more workers to load) but improve throughput for concurrent requests

Configuration Guide

| Use Case | replica | preventCleanup | |----------|-----------|------------------| | High-frequency model** | 4-8 | true | | Occasional use | 1 | false | | Critical path | 2-4 | true | | Background jobs | 1-2 | false |

Memory Considerations

Each worker instance loads the full WebAssembly module into memory:

• Memory per worker = Model required memory (~150-300 MB)
• Total memory = (Number of models) × (Replicas per model) × (Memory per worker)
• Example: 10 models × 3 replicas × 150 MB = ~4500 MB minimum

NOTE: We recommend keeping replica at 2 or fewer, unless you have a very specific performance requirement that justifies higher concurrency.

Cleanup Best Practices

Always call destroy():

const spark = new Spark(config);
try {
  await spark.execute(input);
} finally {
  await spark.destroy();
}

For long-running apps:

• Set offlineMaxService based on available memory
• Use preventCleanup: true sparingly
• Call destroy() before page unload (browsers)

Known Limitations

Browser Memory Constraints

WebAssembly in browsers has strict memory limits:

| Constraint | Limit | Impact | |------------|-------|--------| | WebAssembly memory | ~1-2 GB (browser dependent, varied between platform) | Large models may fail to load | | Total tab memory | ~2-4 GB (browser dependent) | Multiple models + high replicas can hit browser limits |

Recommendations for Browser:

• Limit replica count to 2-4 for large / highly complex models
• Monitor memory usage with browser DevTools
• Consider online-only mode for very large models

Node.js Worker Thread Cost

Each worker thread in Node.js has overhead:

• Startup time: 50-200ms per worker
• Memory overhead: ~150-300 MB per thread (V8 isolate + runtime)
• Efficient thread limit: Typically 50-200 threads (OS dependent)

For Node.js applications:

• Don't create hundreds of workers (replica count × model count should be reasonable)
• Workers are lazy-loaded but all replicas for a model are initialized on first execution

Cross-Service Call (XCall) Depth

Recursion Limit:

• Maximum XCall depth: ~10-20 levels (implementation dependent)
• Deeply nested XCalls increase memory usage and execution time
• Stack overflow possible with excessive recursion

Best Practice:

• Design models to minimize XCall depth
• Avoid circular dependencies between models
• Consider flattening deeply nested call chains

Model Loading Failures

Common offline mode failures:

| Scenario | Browser | Node.js | |----------|---------|---------| | 10+ concurrent replicas | May hit memory limits | Works but slow startup | | Complex XCall graphs | Stack overflow possible | More headroom |

Mitigation:

• Use online mode for exceptionally large models
• Reduce replica count if initialization fails
• Monitor memory consumption during development

Live Demo

See the SDK in action with this interactive CodeSandbox example:

View Demo →