@dloizides/text-generation

Provider-agnostic text / copy generation — one TextGenerator interface over either an in-browser WebLLM engine (runs on WebGPU, no API keys, no server) or a remote OpenAI-compatible API. This is the text seam of the cross-product generation abstraction (Capability Wave C3): consumers depend on the interface, so the backend is swappable without code change.

Why

Both a loaded WebLLM engine and an OpenAI-style API speak the same chat.completions.create({ messages, max_tokens, temperature }) shape. So the only difference between "generate in the browser" and "generate via an API" is which create-function you inject — the generator code is identical.

Install

npm i @dloizides/text-generation

Zero runtime deps. For the in-browser path, the consumer provides @mlc-ai/web-llm (the heavy WebGPU engine) and injects the loaded engine — it is not a dependency of this package.

In-browser (WebLLM)

import {
  ChatCompletionsTextGenerator,
  generateJsonObject,
  isWebGpuSupported,
} from '@dloizides/text-generation';

if (!isWebGpuSupported()) { /* disable the Generate button */ }

const webllm = await import('@mlc-ai/web-llm');
const engine = await webllm.CreateMLCEngine('Llama-3.2-1B-Instruct-q4f16_1-MLC');

const generator = new ChatCompletionsTextGenerator((req) =>
  engine.chat.completions.create(req),
);

const draft = await generateJsonObject(generator, {
  system: 'You are a copywriter. Respond with JSON only.',
  prompt: 'Event: KUCY, Nicosia, 16 May 2026',
});

Remote API (OpenAI-compatible)

import {
  ChatCompletionsTextGenerator,
  createOpenAiChatCompletions,
} from '@dloizides/text-generation';

const create = createOpenAiChatCompletions({
  apiKey: process.env.OPENAI_API_KEY!,   // keep server-side
  model: 'gpt-4o-mini',
});
const generator = new ChatCompletionsTextGenerator(create);
const text = await generator.generate({ system, prompt });

API

• TextGenerator — the seam: generate(request): Promise<string>.
• ChatCompletionsTextGenerator — wraps any OpenAI-shape create function (a WebLLM engine or a remote client).
• createOpenAiChatCompletions(opts) — fetch-based OpenAI-compatible provider (fetchImpl injectable for tests).
• generateJsonObject(generator, request) / parseJsonObject(raw) / stripJsonFences(raw) — JSON-output helpers (tolerant of markdown fences).
• isWebGpuSupported() — cheap synchronous WebGPU gate (safe off-browser).
• TextGenerationError — thrown on unavailable backend / empty / unparseable output.
• Defaults: maxTokens 600, temperature 0.7, OpenAI base URL https://api.openai.com/v1.