cacheai-transformer 🚀

An elegant, production-ready Client-side SDK for Decentralized, Cross-Origin Local AI Weight Sharing.

By sharing a single centralized browser iframe host (https://cc.lemone.online), cacheai-transformer lets developers embed on-device WebGPU models (via Transformers.js and ONNX) into multiple distinct third-party web applications without requiring duplicate downloads of huge model files (e.g., 600MB - 2.5GB).

📦 Why Use Shared Cache?

1. Storage De-duplication: Traditional browsers isolate IndexedDB and Cache Storage by security origin (domain). If a user visits app-a.com and app-b.com which both use local models, they have to download the model twice. By hosting the execution thread inside a single shared iframe running on https://cc.lemone.online, they share the exact same hot-cache state globally.
2. Instant Hot-Starts: Downloading a 1GB model takes minutes. Reusing the cache takes less than 120ms.
3. Fully Private & Client-Side: All calculations run directly on the user's CPU/GPU via WebGPU and WebAssembly. No prompts, audio streams, or history databases are ever sent to remote services.

🛠️ Installation

Install the package via npm:

npm install cacheai-transformer

💻 Usage & Code Examples

1. Initialize Client

The client automatically injects the background sandboxed iframe on instantiation and hooks into bidirectional message streams.

import { CerebralCortexClient } from 'cacheai-transformer';

// Instantiate the SDK client
const client = new CerebralCortexClient({
  sandboxUrl: 'https://cc.lemone.online',
  defaultModel: 'LemOneLabs/Qwen3-0.6B-ONNX'
});

// Explicitly wait for iframe connection if needed (or let methods handle it lazily)
await client.init();

2. Text / Chat Generation

Supports streaming callbacks, nucleolus sampling (top_p), creative temperature adjustments, and multi-turn chat logs.

try {
  const result = await client.generate(
    "How does WebGPU accelerate neural models?",
    {
      temperature: 0.7,
      max_new_tokens: 256,
      thinkEnabled: true // Optional: supports Gemma-4 / reasoning models
    },
    // Optional progress callback (for initial downloads)
    (progress) => {
      console.log(`[${progress.status}] Loaded ${progress.percentage}% of file: ${progress.file}`);
    },
    // Optional streaming token callback
    (token) => {
      process.stdout.write(token);
    }
  );

  console.log("\nFinal Response:", result.response);
  console.log("Benchmarks:", result.performance);
} catch (error) {
  console.error("Inference aborted:", error);
}

3. Vision / Image Description

Describe and analyze images locally without API bills.

const base64Image = "data:image/jpeg;base64,..."; // Or remote URL

const visionResult = await client.generateVision(
  base64Image,
  "Describe this image and identify what objects are in it.",
  {
    modelId: 'onnx-community/Janus-1.3B-ONNX' // Or your chosen vision model
  }
);

console.log("Vision Output:", visionResult.response);

4. Local Music Generation

Generate sound effects or ambient musical themes directly inside the browser.

const musicResult = await client.generateMusic(
  "A quiet, relaxing piano melody with a gentle acoustic guitar accompaniment.",
  {
    modelId: 'Xenova/musicgen-small'
  }
);

// Returns a local Object URL of the compiled WAV audio file
console.log("Audio WAV URL:", musicResult.audio);
console.log("Sample Rate:", musicResult.sampling_rate);

// Create an audio player node dynamically
const audio = new Audio(musicResult.audio);
audio.play();

5. speechToText & textToSpeech

Synthesize audio tracks or transcribe speech streams on-device.

// 1. Text-To-Speech (TTS) Synthesis
const tts = await client.textToSpeech("Decentralized computing is the future of client-side web application designs.");
const audioNode = new Audio(tts.audio);
audioNode.play();

// 2. Speech-To-Text (STT) Transcription
const audioBuffer = new Float32Array([...]); // raw audio sample floats
const stt = await client.speechToText(audioBuffer, {
  modelId: 'onnx-community/whisper-tiny.en'
});
console.log("Transcribed speech text:", stt.response);

6. Utility Functions & Telemetry

Check Cache Contents

Inspect which models are currently cached in IndexedDB:

const cachedModels = await client.getCacheInfo();
console.log("Active caches:", cachedModels);

Clear Cache Weights

Wipe all cached ONNX parameters from disk to reclaim VRAM or storage:

const clearReport = await client.clearCache();
console.log(`Cleared ${clearReport.count} models successfully.`);

Stop Execution

Halt active pipeline operations or weight downloads:

await client.interrupt();

Detect Local Accelerations (Omnix Daemon)

Check if the user is running the ultra-fast local desktop companion daemon (Omnix) to shift execution natively:

const isOmnixLocal = await client.isOmnixRunning();
if (isOmnixLocal) {
  console.log("Omnix desktop daemon is active! Native speed enabled.");
}

⚖️ License

Apache-2.0 License. Designed with ♥ by LemOneLabs.