kanha-ai

Drop-in AI chatbot widget for bots custom-trained on your own website content with Kanha.

The model runs on-device in the visitor's browser over WebGPU. There is no inference API in the loop, so conversations stay on the device and you are not billed per message.

Install

npm install kanha-ai

React is a peer dependency for the React entry point. The CDN widget and Web Component have no framework dependency.

React component

import { KanhaBot } from "kanha-ai";

export default function App() {
  return (
    <KanhaBot
      modelUrl="https://huggingface.co/your-org/your-bot/resolve/main/"
      botName="Acme Assistant"
      welcomeMessage="Ask me anything about Acme."
      suggestions={["What do you sell?", "How does pricing work?"]}
      theme={{ primaryColor: "#0d9488", position: "bottom-right" }}
    />
  );
}

React hook

useKanhaChat gives you the same engine with no UI, so you can build your own. loadProgress is a percentage that restarts on each phase of the first load, so pair it with loadStage ("Downloading model", "Loading cached model", "Preparing GPU", "Almost ready") to explain the number going down.

import { useKanhaChat } from "kanha-ai";

function Chat() {
  const { messages, input, setInput, send, stop, clear, isLoading, mode, loadProgress, loadStage, error } =
    useKanhaChat({ modelUrl: "https://huggingface.co/your-org/your-bot/resolve/main/" });

  return (
    <form onSubmit={(e) => { e.preventDefault(); send(); }}>
      {messages.map((m, i) => <p key={i}><b>{m.role}</b>: {m.content}</p>)}
      <input value={input} onChange={(e) => setInput(e.target.value)} disabled={isLoading} />
    </form>
  );
}

CDN and vanilla JS

<div id="chat"></div>
<script type="module">
  import { mount } from "https://cdn.jsdelivr.net/npm/kanha-ai/dist/widget.js";

  const bot = mount("#chat", {
    modelUrl: "https://huggingface.co/your-org/your-bot/resolve/main/",
    botName: "Acme Assistant",
  });
</script>

mount() returns { stop, clear, destroy } so you can tear the widget down again.

The widget build has no bundler requirement. It pulls the WebGPU runtime at load time from https://cdn.jsdelivr.net/npm/@mlc-ai/web-llm@<version>/+esm, pinned at publish time to the version the package was built against, so a plain <script type="module"> tag is all you need.

Web Component

Loading widget.js registers <kanha-bot> automatically.

<script type="module" src="https://cdn.jsdelivr.net/npm/kanha-ai/dist/widget.js"></script>

<kanha-bot
  model-url="https://huggingface.co/your-org/your-bot/resolve/main/"
  bot-name="Acme Assistant"
  primary-color="#0d9488"
  position="bottom-right"
></kanha-bot>

Props map to kebab-case attributes (modelUrl becomes model-url, ragCorpusUrl becomes rag-corpus-url, repetitionPenalty becomes repetition-penalty). suggestions takes a JSON array string. theme.primaryColor and theme.position are flattened to primary-color and position.

Grounded answers with sources

Point ragCorpusUrl at the grounding corpus published with your bot and the widget retrieves the matching passages before every answer, restricts the model to that context, and renders the source pages under the reply.

<kanha-bot
  model-url="https://huggingface.co/your-org/your-bot/resolve/main/"
  rag-corpus-url="https://huggingface.co/your-org/your-bot/resolve/main/rag-corpus.json"
  bot-name="Acme Assistant"
></kanha-bot>

Grounded corpora use hybrid retrieval by default. The SDK combines BM25 exact-term matches with Xenova/bge-small-en-v1.5 embeddings, then merges both rankings. Embedding runs in a dedicated worker so the chat stays responsive while the corpus is indexed. Corpus vectors are reused in memory by corpus hash and embedding model until the page reloads. Set ragRetrievalMode="bm25" to skip embeddings. Set ragRerankerModel="mixedbread-ai/mxbai-rerank-xsmall-v1" to load the optional cross-encoder reranker.

The hybrid worker defaults to the same pinned kanha-ai package version on jsDelivr, which keeps npm consumer bundles from depending on a copied worker asset. Set retrievalWorkerUrl to the URL where you self-host dist/retrieval-worker.js when CDN loading is not suitable.

If the embedding worker or model cannot load, retrieval continues with BM25 and reports the reason through onRetrieval. The callback also includes component scores, the selected context, final prompt messages, the retrieval runtime identity, prompt budget method, and stage timings. onCompletion receives those submitted messages, the raw model completion, the displayed answer after safeguards, the exact generation settings, timestamps, and a success or error outcome. Failed streaming generations retain any partial raw output without presenting the error bubble as an answer.

When the question is mostly about things the corpus never mentions the bot says so instead of guessing, and no model call is made. Greetings and small talk skip the corpus and use your own system prompt. After the model answers, the widget checks every number, price and percentage in the reply against the retrieved passages and the question; sentences with figures that appear in neither are removed, and an answer with nothing left, or one that says it lacks the information, becomes the standard "I can't answer that from the provided context." with no sources attached. Assistant messages carry the pages they were drawn from as sources, so a custom UI built on useKanhaChat can render its own citations:

{messages.map((m, i) => (
  <div key={i}>
    {m.content}
    {m.sources?.map((s) => <a key={s.url} href={s.url}>{s.title}</a>)}
  </div>
))}

onRetrieveContext takes precedence when both are set.

Before any weights are downloaded, the widget adds up the model's tensor-cache.json and compares the total with the browser storage the visitor has left. If there is not enough room it stops there and says how much the model needs against how much is free, rather than failing part way through the download.

Built-in MiniCPM presets

modelSize: "minicpm5-1b" and modelSize: "minicpm5-2b" load pinned model and WebGPU library files with no modelUrl or modelLib. An explicit modelUrl or modelLib replaces its half of the preset.

<kanha-bot model-size="minicpm5-1b" bot-name="Grounded Assistant"></kanha-bot>

These presets are shared grounded checkpoints, not your bot. For a bot you trained on Kanha, use the modelUrl and modelLib returned for that bot.

Props

| Prop | Type | Default | Description | |------|------|---------|-------------| | modelUrl | string | - | Base URL for your bot's model artifacts | | modelLib | string | auto | URL of the matching WebGPU library file, resolved from modelSize when omitted | | modelSize | string | "small" | Model identity used for runtime selection and metrics; "minicpm5-1b" and "minicpm5-2b" also supply built-in artifacts | | systemPrompt | string | - | System prompt for the bot | | temperature | number | 0.7 | Sampling temperature (0-2) | | topP | number | 0.8 | Top-p sampling threshold | | repetitionPenalty | number | 1.1 | Penalty on already generated tokens, which suppresses repeated phrasing | | maxTokens | number | 1024 | Max tokens generated per response | | enableThinking | boolean | false | Hidden reasoning where the bot supports it | | maxHistoryMessages | number | 12 | Non-system messages sent to the model | | streamUpdateIntervalMs | number | 50 | Minimum delay between visible streaming updates | | cacheBackend | "cache" \| "indexeddb" | "indexeddb" | Browser cache backend for downloaded weights | | workerUrl | string \| URL | - | Run inference in a dedicated Web Worker | | retrievalWorkerUrl | string \| URL | pinned SDK worker | Self-hosted hybrid retrieval worker URL | | contextWindowSize | number | - | Context window override | | onMetrics | (m: KanhaChatMetrics) => void | - | Token usage and latency callback | | onRetrieveContext | (query: string) => Promise<string \| null> | - | Retrieval hook called before generating | | ragPromptTemplate | string | - | Template for retrieved context, with {context} and {query} | | ragCorpusUrl | string | - | Grounding corpus to retrieve from, which also turns on source links | | ragRetrievalMode | "hybrid" \| "bm25" | "hybrid" | Hybrid semantic and keyword retrieval, or keyword-only fallback | | ragEmbeddingModel | string | "Xenova/bge-small-en-v1.5" | Transformers.js embedding model used by hybrid retrieval | | ragRerankerModel | string | - | Optional Transformers.js cross-encoder used to rerank candidates | | onRetrievalProgress | (p: RetrievalProgress) => void | - | Model download, corpus indexing, and reranking progress callback | | onRetrieval | (e: RetrievalEvent) => void | - | Called for every grounded turn with the decision (grounded, no_match, greeting), the passages picked with their scores, and the context length. Set it as a JS property on <kanha-bot>, it has no attribute | | onCompletion | (e: CompletionEvent) => void | - | Called with raw and displayed answers, submitted messages, generation settings, and timestamps | | botName | string | "AI Assistant" | Display name (widget only) | | welcomeMessage | string | "Ask me anything!" | Empty-state message (widget only) | | suggestions | string[] | [] | Suggested prompts (widget only) | | theme | { primaryColor?, position? } | teal, bottom-right | Widget theming |

minRamGb and weightBytes are deprecated and ignored.

Optional entry points

| Import | What it gives you | |--------|-------------------| | kanha-ai | KanhaBot, useKanhaChat, types | | kanha-ai/widget | mount() and the <kanha-bot> element, no React | | kanha-ai/rag | LocalRAG for in-browser retrieval over your own documents | | kanha-ai/worker | Prebuilt Web Worker script to pass as workerUrl |

@huggingface/transformers is installed with the SDK for grounded token counting and local retrieval. Pair LocalRAG with onRetrieveContext to ground answers in documents you supply at runtime.

Browser requirements

WebGPU is required. That means a recent Chrome, Edge, or Chromium-based browser, Safari 18+, or Firefox with WebGPU enabled. Model weights download once and are cached in the browser, so the first message is slower than the rest.

Docs

Full setup guide, including how to get your bot's model URLs: kanha.ai/docs/sdk

License

MIT