@molecule/api-ai-embeddings-local

Local (offline) ai-embeddings provider for molecule.dev.

Runs a small sentence-embedding model (default bge-small-en-v1.5, 384-dim) in a worker thread via Transformers.js (onnxruntime) — no API key, no per-call cost, and no network at query time. Bond it once at startup, then use the @molecule/api-ai-embeddings core anywhere.

Auto-generated, AI-first package reference for the molecule.dev ecosystem. It is written to be read by coding agents as much as by people, and is generated from this package's source — edit src/index.ts JSDoc, not this file.

Quick Start

import { setProvider } from '@molecule/api-ai-embeddings'
import { provider } from '@molecule/api-ai-embeddings-local'

setProvider(provider) // at startup

// anywhere after:
import { requireProvider } from '@molecule/api-ai-embeddings'
const vector = await requireProvider().embedQuery('some text') // number[] (384 dims)
const vectors = await requireProvider().embedDocuments(['a', 'b']) // number[][]

Type

provider

Installation

npm install @molecule/api-ai-embeddings-local @huggingface/transformers @molecule/api-ai-embeddings

API

Interfaces

LocalEmbeddingsProvider

Local provider lifecycle: close releases its model worker and rejects pending calls.

interface LocalEmbeddingsProvider extends AIEmbeddingsProvider {
  close(): Promise<void>
}

LocalEmbeddingsConfig

Configuration for the local embeddings provider. Every field is optional and has an env-var fallback, so the provider works with zero configuration.

interface LocalEmbeddingsConfig {
  /**
   * Model id (Transformers.js / HuggingFace). Defaults to `Xenova/bge-small-en-v1.5`
   * (384-dim) or the `MOL_EMBEDDINGS_LOCAL_MODEL` env var.
   */
  model?: string
  /** Pooling strategy. Defaults to `cls` — bge models are trained for CLS pooling. */
  pooling?: LocalEmbeddingsPooling
  /** L2-normalize outputs so a dot product equals cosine similarity. Defaults to `true`. */
  normalize?: boolean
  /**
   * Directory Transformers.js caches downloaded weights in (or the
   * `MOL_EMBEDDINGS_LOCAL_CACHE_DIR` env var). Use a persistent path so the
   * one-time download survives restarts.
   */
  cacheDir?: string
  /**
   * Directory holding a pre-bundled model for fully-offline / air-gapped use (or
   * the `MOL_EMBEDDINGS_LOCAL_MODEL_PATH` env var). Setting it disables remote
   * fetch unless {@link allowRemoteModels} is explicitly `true`.
   */
  localModelPath?: string
  /**
   * Allow downloading the model from HuggingFace on first use. Defaults to `true`,
   * or `false` when {@link localModelPath} is set.
   */
  allowRemoteModels?: boolean
  /**
   * How many texts to run through the model per forward pass (or the
   * `MOL_EMBEDDINGS_LOCAL_BATCH_SIZE` env var). Defaults to 32.
   *
   * This is a MEMORY bound, not a throughput knob. Inference is in-process, so
   * the whole batch's activations are resident at once and attention allocates
   * on the order of `batch × heads × sequence²`. Handing the model an entire
   * corpus in one call therefore scales peak RSS with the corpus: indexing 898
   * documents unbatched peaked at ~3.8 GiB and was OOM-killed under a 1–2 GiB
   * container limit — and since the kernel delivers that kill, no `catch` in
   * the calling code can degrade gracefully.
   *
   * Raise it if you have headroom and want fewer, larger passes; lower it for a
   * tighter memory ceiling. Values below 1 are clamped to 1.
   */
  batchSize?: number
  /**
   * ONNX threads per inference (or MOL_EMBEDDINGS_LOCAL_THREADS). Defaults to 1,
   * leaving other cores available to the host API. One worker serializes all
   * requests for this provider, including concurrent document/query calls.
   */
  inferenceThreads?: number
  /**
   * Maximum request lifetime including queue time (or MOL_EMBEDDINGS_LOCAL_TIMEOUT_MS).
   * Defaults to 300000 ms. Expiry terminates the worker and rejects every pending
   * call; the next call can start a fresh worker. Non-positive values use the default.
   */
  requestTimeoutMs?: number
}

Types

LocalEmbeddingsPooling

Pooling strategy applied to the model's token embeddings to produce one vector per input.

type LocalEmbeddingsPooling = 'cls' | 'mean' | 'none'

Functions

createProvider(config)

Create a local embeddings provider. Config falls back to env vars, so createProvider() with no arguments works out of the box.

function createProvider(config?: LocalEmbeddingsConfig): LocalEmbeddingsProvider

• config — Optional model / pooling / cache configuration.

Returns: A {@link LocalEmbeddingsProvider} backed by a worker-hosted ONNX model.

Constants

provider

The provider implementation. Bond it with the ai-embeddings core's setProvider. Model loading is deferred until the first embedding call.

const provider: LocalEmbeddingsProvider

Core Interface

Implements @molecule/api-ai-embeddings interface.

Bond Wiring

Setup function to register this provider with the core interface:

import { setProvider } from '@molecule/api-ai-embeddings'
import { provider } from '@molecule/api-ai-embeddings-local'

export function setupAiEmbeddingsLocal(): void {
  setProvider(provider)
}

Injection Notes

Requirements

Peer dependencies:

• @molecule/api-ai-embeddings >=1.0.1

Runtime Dependencies

• @huggingface/transformers

• @molecule/api-ai-embeddings

• The model loads lazily on the first embed call (~a few seconds) and then stays resident (~200–300 MB RAM). Nothing loads if you never embed.

• Model loading, tokenization and inference run off the caller's event loop. One worker per provider serializes requests, with one ONNX inference thread by default. Call close() to release it; an idle worker does not keep Node alive. Requests time out after five minutes (including queue time), rejecting pending work and allowing the next call to retry with a fresh worker. Configure inferenceThreads / requestTimeoutMs or MOL_EMBEDDINGS_LOCAL_THREADS / MOL_EMBEDDINGS_LOCAL_TIMEOUT_MS to override those bounds.

• First use downloads the model (~34 MB) and caches it. For fully-offline / air-gapped deployments, bundle the model and set localModelPath (or the MOL_EMBEDDINGS_LOCAL_MODEL_PATH env var) — that disables the remote fetch.

• Configure via createProvider({ model, pooling, cacheDir, localModelPath }) or the MOL_EMBEDDINGS_LOCAL_* env vars. Outputs are L2-normalized, so a dot product equals cosine similarity.

• Pulls @huggingface/transformers + onnxruntime-node (~350 MB installed) — a real third-party dependency, unlike most @molecule/* packages. Add it only where you actually embed.

E2E Tests

Integration checklist — drive the real flow (no mocks), adapt each item to this app's actual data and features, and check every box off one by one. A box you can't check is an integration bug to fix — not a skip. Embeddings are infrastructure, so PROVE them through the feature they power (semantic search / "related items" / dedup) AND with a direct property check on the vectors:

• [ ] embedQuery(text) returns a non-empty numeric number[] of the model's fixed dimension, and every vector from embed/embedDocuments has that SAME length — no empty arrays, no NaN/null entries, and the length is identical across calls (a query and a document must be comparable).
• [ ] The semantic property holds — there is NO built-in similarity helper, so compute cosine similarity yourself (dot product over the two magnitudes): two related texts ("dog"/"puppy", or a query and a matching doc) score HIGH, two unrelated texts ("dog"/"quarterly taxes") score clearly LOWER. If every pair scores alike the vectors are dead — one positive check alone doesn't prove it.
• [ ] Ranking works: embed a query plus a handful of documents and sort by cosine similarity — the semantically closest document ranks ABOVE the unrelated ones. This ordering is the whole point; a query that ranks an off-topic doc first is broken.
• [ ] The feature built on it works end-to-end from the real UI: semantic search / "related items" / dedup returns results ranked by MEANING, not keyword — a search for a synonym or paraphrase finds the right item even when it shares NO words with the query (the case a plain text/keyword search would miss).
• [ ] Embedding is stable: the same text embedded twice yields (near-)identical vectors, so a stored index stays valid — re-embedding an item must not silently drift it out of its own neighborhood.
• [ ] Batching is order-preserving: embedDocuments([a, b, c]) (or embed({ input })) returns exactly one vector per input in the SAME order — embeddings[i] is the vector for input[i], never shuffled, merged, or dropped.
• [ ] Embedding runs SERVER-SIDE: the call goes through the app's API and the provider key never reaches the browser (no provider request or key in the Network tab). Any endpoint that embeds caller-supplied text is authenticated and rate-limited — an open embed endpoint is an unbounded per-token cost/abuse vector.