@axiomantic/nimts

v0.1.0

Published

a month ago

Seamless TypeScript & WebAssembly bridge for Nim with Vite integration, sync/async loaders, and Future<->Promise bridge

0High
0Medium
0Low

elijahr

nim typescript wasm webassembly vite bindings

NimTS: Seamless TypeScript & WebAssembly Bridge for Nim

NimTS is a zero-friction bridge generator connecting Nim to TypeScript and WebAssembly (WASM).

📖 Full Documentation & Interactive Guides: https://axiomantic.github.io/nimts/

NimTS enables writing high-performance WebAssembly libraries in Nim with natural Nim signatures, companion C-ABI entry points, automatic .d.ts declaration generation, and modern TypeScript wrappers.

Key Highlights

⚡ Zero-Overhead {.exportts.}: Preserves native Nim signatures and behavior. Procs remain 100% callable by native Nim CLI tools, VST/CLAP audio plugins, and native unit tests with zero overhead.
🔄 Bidirectional Async Bridge: Automatically bridges Nim Future[T] and JavaScript Promise<T> without blocking the main browser thread.
🎛️ Stateful Handles with GC: Exports ref object instances as type-safe opaque handles with automatic garbage collection via JavaScript's FinalizationRegistry and deterministic manual .dispose().
🔁 Closure Iterators & SPSC Channels: Stream data from Nim closures as standard ES2015 IterableIterator<T> (for..of, [...]) and lock-free SPSC channels as AsyncIterable<T> (for await..of).
📦 Vite Plugin: Import .nim files directly into your frontend code with on-the-fly compilation and hot module reloading.
🎵 AudioWorklet & Worker Ready: Supports streaming instantiation, pre-compiled WebAssembly.Module sharing, and raw BufferSource buffers.

Installation

In your Nim project (Nimble)

nimble install nimts

In your TypeScript / Vite project (npm)

npm install @axiomantic/nimts

Note: For full-stack applications (e.g. Vite + React/Vue with .nim source files), install both: Nimble provides the compiler tools & macros, while npm provides the Vite plugin (@axiomantic/nimts/vite) and runtime (@axiomantic/nimts/runtime).

Quickstart (3 Minutes)

1. Write your Nim module (`engine.nim`)

# engine.nim
import std/asyncdispatch
import nimts

type
  AudioEngine* {.exportts.} = ref object
    sampleRate*: float32
    volume*: float32

  SynthPatch* = object
    name*: string
    polyphony*: int32

# 1. Primitives: passed & returned directly
proc addNumbers*(a: int32, b: int32): int32 {.exportts.} =
  a + b

# 2. Strings: UTF-8 marshaled automatically
proc greet*(name: string): string {.exportts.} =
  "Hello, " & name & " from Nim!"

# 3. Binary buffers: maps directly to Uint8Array
proc processData*(data: openArray[byte]): seq[byte] {.exportts.} =
  result = newSeq[byte](data.len)
  for i in 0 ..< data.len:
    result[i] = data[i] xor 0xFF'u8

# 4. Stateful handles for ref object instances
proc createEngine*(sampleRate: float32): AudioEngine {.exportts.} =
  AudioEngine(sampleRate: sampleRate, volume: 1.0'f32)

proc setVolume*(engine: AudioEngine, vol: float32) {.exportts.} =
  if engine != nil:
    engine.volume = vol

# 5. Non-blocking async Future[T] -> JS Promise<T>
proc computeAsync*(x: int32): Future[int32] {.async, exportts.} =
  return x * 42

static:
  # Emits engine.d.ts and engine.js glue code at compile-time
  emitTsGlueFiles("engine", ".")

2. Compile to WebAssembly

Using the standalone CLI:

nimts compile engine.nim

Or using the bundled compilation script:

./scripts/compile_wasm.sh engine.nim engine.wasm

3. Consume in TypeScript / JavaScript

// app.ts
import {
  initNimTs,
  addNumbers,
  greet,
  processData,
  createEngine,
  setVolume,
  computeAsync
} from './engine.js';

// Initialize the WebAssembly module
await initNimTs();

// Call synchronous functions
console.log(addNumbers(10, 32)); // 42
console.log(greet("TypeScript")); // "Hello, TypeScript from Nim!"

// Process binary buffers
const input = new Uint8Array([0x00, 0xAA, 0xFF]);
const output = processData(input); // Uint8Array [ 255, 85, 0 ]

// Stateful instances with handles
const engine = createEngine(48000.0);
setVolume(engine, 0.8);
engine.dispose(); // Explicit release, or automatic GC via FinalizationRegistry

// Async Future[T] seamlessly awaited as Promise<T>
const result = await computeAsync(10); // 420

Solved WebAssembly Gotchas

Gotcha 1: Chrome / V8 4KB Main-Thread Sync Compile Limit

Browser main threads disallow synchronous WebAssembly compilation (WebAssembly.compile or new WebAssembly.Module) if the binary buffer size exceeds 4KB, throwing:

RangeError: WebAssembly.Compile is disallowed on the main thread, if the buffer size is larger than 4KB

NimTS Solution: The generated JavaScript wrapper automatically handles all modern runtime environments:

WebAssembly.instantiateStreaming(fetch(...)) for modern browsers and bundlers.
Explicit initNimTs(options) accepting:
- wasmPath: URL or local file path.
- wasmModule: Pre-compiled WebAssembly.Module (ideal for passing across threads to an AudioWorkletNode or Web Worker).
- wasmBytes: Raw BufferSource / ArrayBuffer.
- wasmInstance: Existing instantiated WebAssembly.Instance.

Gotcha 2: Nim async/await (`Future[T]`) $\leftrightarrow$ JavaScript `Promise<T>` Bridge

WebAssembly is fundamentally single-threaded; blocking the WASM thread while waiting for an asynchronous event would freeze the browser host.

NimTS Solution: NimTS implements a bidirectional, non-blocking promise bridge:

Nim $\to$ JS: When an exported proc foo(): Future[T] {.async, exportts.} is called, Nim allocates a monotonic PromiseId, registers a completion callback, and immediately returns the ID to JavaScript, which wraps it in a real Promise<T>. When the Future completes or fails, Nim notifies JavaScript via nimts_js_resolve_promise or nimts_js_reject_promise.
JS $\to$ Nim: When Nim awaits an external async operation, it registers a pending Future[T] and passes a FutureId. When the JS promise settles, JS calls back into WASM wasm.nimts_resume_future(futureId, dataPtr, dataLen, isError) which immediately wakes the waiting Nim coroutine.

Gotcha 3: Stateful Objects & Handle Leaks

Passing raw pointers between WebAssembly linear memory and JavaScript risks use-after-free, memory corruption, and garbage collection leaks.

NimTS Solution: Stateful entities (ref object) are registered in an internal Nim handle registry that increments Nim GC reference counts (GC_ref). JavaScript receives a typed class holding an opaque HandleId:

Manual cleanup: .dispose() deterministically frees the handle and unrefs the Nim object (GC_unref).
Automatic fallback: Bound to JavaScript's modern FinalizationRegistry to prevent leaks if .dispose() is forgotten.

Type Mapping Reference

Vite Plugin (`vite-plugin-nimts`)

Import .nim files directly in your Vite application:

// vite.config.ts
import { defineConfig } from 'vite';
import { nimtsPlugin } from '@axiomantic/nimts/vite';

export default defineConfig({
  plugins: [
    nimtsPlugin({
      extraNimFlags: ['-d:release']
    })
  ]
});

Then in your frontend:

import { initNimTs, processData } from './engine.nim';

await initNimTs();
const result = processData(new Uint8Array([1, 2, 3]));

AudioWorklet / Web Worker Pattern

To run Nim in an AudioWorkletProcessor:

// main.ts
const wasmModule = await WebAssembly.compileStreaming(fetch('/engine.wasm'));
audioContext.audioWorklet.addModule('dsp-processor.js');

const node = new AudioWorkletNode(audioContext, 'dsp-processor', {
  processorOptions: { wasmModule }
});

// dsp-processor.ts
import { initNimTs, processAudio } from './engine.js';

class DspProcessor extends AudioWorkletProcessor {
  constructor(options) {
    super();
    // Initialize synchronously from pre-compiled module
    initNimTs({ wasmModule: options.processorOptions.wasmModule });
  }

  process(inputs, outputs) {
    // Zero-allocation high-speed DSP processing
    processAudio(inputs[0][0], outputs[0][0]);
    return true;
  }
}
registerProcessor('dsp-processor', DspProcessor);

CLI Reference

NimTS includes a command-line tool for compiling Nim modules into WebAssembly:

# Display help and available backends
nimts --help

# Display version
nimts --version

# Compile a Nim module using the standalone micro-WASM backend (default)
nimts compile src/dsp.nim dist/dsp.wasm

# Explicitly choose the standalone micro-WASM backend (LLVM clang + wasm-ld + wasi-sysroot)
nimts compile src/dsp.nim dist/dsp.wasm --backend:standalone

# Choose the Emscripten backend (emcc with POSIX shims, virtual FS, and environment polyfills)
nimts compile src/dsp.nim dist/dsp.wasm --backend:emscripten

Backend Comparison

Running Tests

NimTS has comprehensive test suites for both native Nim execution and WebAssembly runtime integration:

# Run Nimble unit tests (macro non-invasiveness, memory, handles, async bridge, codegen, CLI)
nimble test

# Run TypeScript / Vitest WebAssembly integration tests
npm test

Publishing

To publish a new version to npm and GitHub:

# 1. Ensure all tests pass and package builds cleanly
npm run prepublishOnly

# 2. Check packaging contents
npm pack --dry-run

# 3. Publish to npm
npm publish --access public

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for local development setup, toolchain requirements, and guidelines.