Startin'blox AI Bridge

Introduction

The solid-ai-bridge component provides a client-side AI inference bridge for Startin'blox applications. It leverages Hugging Face Transformers for tokenization and ONNX Runtime Web for executing pre-trained ONNX models directly in the browser. This enables features like Named Entity Recognition (NER) to filter and process data based on natural language prompts, enhancing the interactivity and intelligence of your Solid applications.

Features

• Client-side AI Inference: Runs AI models directly in the browser, reducing server load and improving response times.
• Flexible Model Integration: Supports both local (ONNX Runtime Web, Hugging Face Transformers) and server-side AI models.
• Custom Model Logic: Allows registration of custom functions for local model inference.
• Named Entity Recognition (NER): Utilizes pre-trained ONNX models to identify and extract entities from natural language prompts.
• Hugging Face Transformers Integration: Seamlessly integrates with @huggingface/transformers for efficient text tokenization.
• ONNX Runtime Web Support: Executes ONNX (Open Neural Network Exchange) models for various AI tasks.
• Custom Event API: Provides a window.sibAIBridge.aiQuery function and listens for ai-query custom events, dispatching ai-result with processed data including datas, model, and prompt.
• Resource Filtering: Filters Solid resources based on entities extracted from user prompts and provided labels.

Getting started

Installation

npm install
npm run watch

Usage

The solid-ai-bridge component requires models for Named Entity Recognition (NER). You can add models using the addModel method:

await window.sibAIBridge.addModel(
  "YourModelName",
  {
    local: true, // or false for a server-side model
    // local models parameters:
    pretrained_model_path: "https://cdn.startinblox.com/models/3dobjects/search/models/ner_tokenizer", // Required for local models
    model_uri: "https://cdn.startinblox.com/models/3dobjects/search/models/ner_model_quantized.onnx", // Required for local models
    register: async (pretrained_model_path: string, model_uri: string) => {
      // Your registration logic here, e.g.,
      // const tokenizer = await AutoTokenizer.from_pretrained(pretrained_model_path);
      // const session = await InferenceSession.create(model_uri);
      // return [tokenizer, session];
      throw new Error("Register function not implemented for example");
    },
    // or, server-side models parameters:
    server: "http://localhost:8000/summarize", // Required for server-side models
  },
  // Only for local models:
  async (model, datas, prompt) => {
    // Your model application logic here
    return datas;
  }
);

To perform an AI query, you can either use the window.sibAIBridge.aiQuery function directly or dispatch a custom ai-query event.

Using window.sibAIBridge.aiQuery:

const result = await window.sibAIBridge.aiQuery(
  "3dobject-search", // or your custom model name
  [/* array of Resource objects */],
  "your prompt here", // optional
);
console.log(result.datas); // Filtered resources

Using ai-query and ai-result events:

Dispatch an ai-query event with the necessary details:

window.dispatchEvent(
  new CustomEvent("ai-query", {
    detail: {
      id: "unique-query-id",
      prompt: "your prompt here",
      datas: [/* array of Resource objects */],
      model: "3dobjects-search", // or your custom model name
    },
  })
);

Listen for the ai-result event to get the processed data:

window.addEventListener("ai-result", (event) => {
  const { id, prompt, datas, model, error } = event.detail;
  if (error) {
    console.error(`AI Query Error for ID ${id}:`, error);
  } else {
    console.log(`AI Query Result for ID ${id}:`, datas);
  }
});

License

MIT