@dromio/inference

Provider-neutral contracts for embeddings, OCR, transcription, provider lifecycle, explicit capability routes, normalized usage, and attempt events.

Embedding spaces

An embedding is compatible with an index only when the complete EmbeddingSpaceIdentity matches. Dimensions alone never establish compatibility. The identity records the provider/model revision, dimensions, preprocessing contract, and an explicit compatibility ID.

const result = await inference.embedMany({
  inputs: [{ text: "first memory" }, { text: "second memory" }],
});

console.log(result.results[0]?.space.compatibilityId);

Explicit primary routes

The router executes one selected provider for each capability. It never switches providers automatically. Typed transient failures may retry the same provider within a bounded retry policy.

const router = createInferenceRouter({ providers, store });

await router.activate({
  capability: "embedding",
  embeddingSpace,
  expectedRevision: 3,
  model: "text-embedding-3-small",
  processingLocation: "hosted",
  providerId: "openai",
  retry: { backoffMs: 250, maxAttempts: 2 },
});

Applications own durable InferenceRouteStore implementations, operator authorization, provider policy, credential resolution, audit persistence, and index migration. Provider packages own vendor protocol mapping.

flowchart LR
  App["Application policy"] --> Store["Durable route store"]
  Store --> Router["Explicit primary router"]
  Router --> Selected["Selected provider"]
  Router -. "same-provider retry" .-> Selected