OpenVINO™ GenAI Node.js bindings

Use OpenVINO GenAI to deploy most popular Generative AI model pipelines that run on top of highly performant OpenVINO Runtime.

Quick Start

Install the openvino-genai-node package:

npm install openvino-genai-node

Use the openvino-genai-node package:

import { LLMPipeline, ChatHistory } from "openvino-genai-node";

const pipe = await LLMPipeline(MODEL_PATH, device);

const input = "What is the meaning of life?";
const config = { max_new_tokens: 100 };

const chatHistory = new ChatHistory();
chatHistory.push({ role: "user", content: input });
const decodedResults = await pipe.generate(chatHistory, config, streamer);
chatHistory.push({ role: "assistant", content: decodedResults.toString() });

// Output all generation result
console.log(decodedResults.toString());

function streamer(subword) {
  process.stdout.write(subword);
}

Supported Platforms

• Windows x86
• Linux x86/ARM
• MacOS x86/ARM

Build From Sources

Build OpenVINO™ GenAI JavaScript Bindings from sources following the instructions.

License

The OpenVINO™ GenAI repository is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.