@bebeal/data-evals

TypeScript framework for data evals with WebAssembly support. Write compute-heavy logic in C -> compile to WASM -> call from TypeScript

Quick start

npm install
npm run build
npm run test

Expected output

=== TypeScript Eval ===
Input: 1200
Computed: 419126400
Metrics: { matrixSum: 419126400 }
Time: 2976.38ms
✓ TypeScript test passed

=== WASM Eval ===
Input: 1200
Computed: 419126400
Metrics: { matrixSum: 419126400 }
Time: 1594.09ms
✓ WASM test passed

✅ WASM is 1.87x faster

Usage

TypeScript Eval

import { TsEval } from '@bebeal/data-evals';

class MyEval extends TsEval<string, number, { score: number }> {
  compute(input: string): number {
    return input.length;
  }

  analyze(input: string, computed: number): { score: number } {
    return { score: computed * 2 };
  }

  render(input: string, computed: number, metrics: { score: number }): string {
    return `<div>Length: ${computed}, Score: ${metrics.score}</div>`;
  }
}

const eval = new MyEval();
const result = eval.run("hello"); // { computed: 5, metrics: { score: 10 }, html: "..." }

WASM Eval

import { WasmEval } from '@bebeal/data-evals';

class MyWasmEval extends WasmEval<string, number, { score: number }> {
  render(input: string, computed: number, metrics: { score: number }): string {
    return `<div>Result: ${computed}</div>`;
  }
}

const eval = new MyWasmEval();
await eval.init('./my_wasm.js');
const result = eval.run("test");

C Interface

Default: WasmEval uses ccall with string I/O. C functions take char* input and return char* output:

char* compute(char* input) {
    // Your code here
    return result_string;
}

char* analyze(char* input, char* computed) {
    // Return JSON string
    return "{\"score\":42}";
}

emcc file.c -o wasm.js \
  -s EXPORTED_FUNCTIONS=["_compute","_analyze"] \
  -s EXPORTED_RUNTIME_METHODS=["ccall"] \
  -s MODULARIZE=1 -O3

Custom: Override compute(), analyze(), or render() to interact with WASM however you want

Requirements

WASM tests require Emscripten:

# macOS
brew install emscripten

# Linux
apt install emscripten        # Ubuntu/Debian
dnf install emscripten        # Fedora
pacman -S emscripten          # Arch

# Windows
choco install emscripten      # or: winget install emscripten

# Any platform (emsdk)
git clone https://github.com/emscripten-core/emsdk.git && cd emsdk
./emsdk install latest && ./emsdk activate latest && source ./emsdk_env.sh