@aria-framework/ai

A model seam shared by apps: one client (createAiClient) over several providers, plus the writing engines (Polish, Generate), a fact-preservation guard, an untrusted-text fence, and the stores and views behind an AI admin page. Prompts, routing policy and credentials stay in the app that uses the package.

| Provider | provider value | Address | |---|---|---| | LM Studio | lmstudio | baseUrl (default http://localhost:1234/v1) | | Any OpenAI-compatible server | openai-compatible | baseUrl | | Anthropic | anthropic | https://api.anthropic.com/v1 | | lmx supervised stack | lmx | discovered at run time, never configured |

npm install @aria-framework/ai undici     # undici only if you use lmx with a pinned certificate

The client

const { createAiClient } = require('@aria-framework/ai');

const ai = createAiClient({
  // Called on every request, so a settings change needs no restart.
  resolveConfig: async () => ({
    enabled: true, provider: 'openai-compatible',
    baseUrl: 'http://gpu-box:8080/v1', model: 'qwen3', apiKey: '...',
    timeoutMs: 60000, maxTokens: 1024, contextTokens: 8192
  }),
  budget: { async assertWithinBudget(cfg, meta) {}, async record(cfg, result, meta) {} }, // optional
  //        recordEmbedding(cfg, result, meta) is optional too: embedding usage, report only (0.29.0)
  logger: console                                                                        // optional
});

const r = await ai.complete({
  system, messages, maxTokens: 400,
  schema,                 // optional: constrained JSON
  // reasoning: 'full',   // optional, every provider: 'off' (default) or 'full' (see Reasoning below)
});
// r.text, r.json (when schema), r.model, r.usage.total, r.ms

Every failure is thrown as an AiError with a kind: disabled, unconfigured, unreachable, timeout, auth, rate_limit, bad_response or refused. The message is written for an admin screen and never includes a URL or key.

A failed call that spent tokens is still metered (0.26.1). When the model answered but the answer was unusable (it thought until the limit, or a structured reply was cut off), the error carries usage ({ prompt, completion, total }) and budget.record(cfg, { usage, model, ms, failed: true }, meta) is called before the error is thrown. A failure that spent nothing (refused, unreachable, timeout) records nothing. If record throws, the original error still reaches the caller. The error also carries model, the model the server says answered, and record receives that (0.27.1); it falls back to the configured model only when the reply named none. On lmx the configured model is empty. A caller that metered err.usage itself before 0.26.1 must stop, or the call is counted twice.

providerStore, usageStore, speedStore and lmxStore are optional db-worker-backed stores. Each exports a schemaFor(dialect), so the app's migration can be checked against it.

lmx

lmx is a supervisor that runs llama.cpp engines and publishes a status document saying which engines exist, which model each one runs and what state it is in. The full contract is CLIENT.md in the lmx repo, and this adapter implements it. The rules you would otherwise have to write yourself:

| Contract rule | What the adapter does | |---|---| | Poll /status about every 2 s | One poller per stack (keyed by instance), shared by every engine on it, unref()'d. Each read is bounded (statusTimeoutMs, default 5 s), concurrent reads share one fetch, and pollMs is floored at 500 ms | | Route only to healthy; draining gets nothing new | Selects for healthy, so a state lmx adds later can never receive work by accident | | Engine URLs come from the document | Resolved on every call, and never stored | | A status outage is not an inference outage | Keeps routing on the last good document for 3 minutes (staleMs), logging loudly | | Pin the certificate; never disable verification | Uses an undici Agent({ connect: { ca } }) per stack. Never NODE_EXTRA_CA_CERTS, never rejectUnauthorized:false | | Choose the reasoning flag from the model | By default thinking is kept to a minimum: qwen → chat_template_kwargs.enable_thinking=false, gpt-oss → reasoning_effort:'low', anything else gets neither (with a warning). reasoning: 'full' on the call (since 0.26.0) sends enable_thinking=true / reasoning_effort:'high' instead. The flag always overrides a raw extra; only the named option changes it. Since 0.27.0 the same rule applies to every provider. See Reasoning | | Size requests against maxInputTokens (per slot) | contextTokens is taken from the engine, not from config | | Identity is (instance, id), falling back to name | lmx.engineId is matched first; rekeyPlan() migrates rows when ids are minted or engines renamed | | A 429 is about the key, not the engine | Fails fast (since 0.25.0): throws rate_limit with lmxSkip: 'lmx_throttled' and retryAfterMs. The client never waits; whether to wait or move on is the caller's decision | | The document's instance must match | A document for another instance is refused, because engine names collide across stacks. lmxVerify then returns engines: null, so nothing downstream (rekeyPlan) can act on another stack's list |

Ask the operator for

• Direct to lmx: the status URL (https://<host>:9443/status), lmx.crt, status.token, and an engine key (lmx keygen --add). Never take admin.token.
• Through the gateway: the status URL https://<host>/e/status, the certificate if it is self-signed, and one infer-scoped lmxk_… key. That one key is both the status token and the engines key, so put the same value in both fields.

The adapter takes the status URL exactly as given; it does not append /status.

Config shape

resolveConfig() (or the cfgOverride you pass to complete) returns:

{
  enabled: true,
  provider: 'lmx',
  apiKey: '<engines key>',           // sent to the ENGINE as Bearer
  timeoutMs: 60000, maxTokens: 1024,
  logger,                            // optional, gets the "status unreachable" warnings
  lmx: {
    instance: 'site-a-dev',          // MUST equal the document's top-level `instance`
    statusUrl: 'https://lmx-host:9443/status',   // or https://gateway/e/status
    statusToken: '<status token>',   // the same lmxk_ key as apiKey through the gateway
    ca: '-----BEGIN CERTIFICATE-----…',          // PEM to pin; null = system trust store
    engine: 'advanced',              // engine name: always stored, used for display and logs
    engineId: '7c9e6679-…',          // engine id when the stack publishes one; null before mint-ids
    pollMs: 2000, staleMs: 180000,   // optional; pollMs is floored at 500, null/undefined = default
    statusTimeoutMs: 5000            // optional; one status read's deadline
  }
}

model is not needed. The engine reports which model it runs, and anything you send is ignored.

What to store

The operator assigns engines to jobs, and the supervisor never does. For each choice, store:

• instance, which is the stack's own name, checked against the document.
• engine_id, when present. It is the only field that survives a rename.
• engine (the name), as the fallback before mint-ids and for display.

Do not store the URL, port, model, or label. The label is for display only and should be re-read on every poll.

providerStore.schemaFor() has lmx_instance, lmx_engine and lmx_engine_id. lmxStore.schemaFor() holds the supervisor row (id = instance, status_url, poll and stale intervals). Credentials belong in your own secret store; this package never sees one at rest.

Re-keying when ids appear or engines are renamed

Call this after each status read, for example from the same refresh that updates labels:

const { rekeyPlan } = require('@aria-framework/ai');
for (const step of rekeyPlan(rows, doc.engines)) {
  // { id: 'row-id', set: { lmx_engine_id: 'uuid' }, why: 'minted' }
  // { id: 'row-id', set: { lmx_engine: 'new-name' }, why: 'renamed', from: 'old-name' }
  await updateRow(step.id, step.set);   // and audit it
}

minted means the same engine has gained an id, so its routes, health and history carry over to the id. A row whose id has disappeared is left alone, and the panel shows it as missing. It is never re-pointed at whichever engine now has its old name.

To find a stored row's engine in a status document, use the same rule the router, the panel and the verifier use. Import it from the package root, not from providers/lmxDiscovery:

const { findEngine } = require('@aria-framework/ai');
const engine = findEngine(doc.engines, { id: row.lmx_engine_id || null, name: row.lmx_engine });

Rotating credentials, and removing a stack

The poller takes its token, certificate and intervals from the config you pass on every call (discoveryFor reconfigures the live poller since 0.25.0), so a rotated token or a replaced certificate takes effect on the next call — no restart. A changed statusUrl replaces the poller. When you delete or disable a supervisor, call require('@aria-framework/ai/providers/lmx').forgetDiscovery(instance) so its poller stops.

Errors and routing

When an lmx call cannot run, the thrown AiError carries err.lmxSkip. None of these mean the engine is broken, so do not trip a circuit breaker on them. Move to the next endpoint instead:

| lmxSkip | Meaning | |---|---| | lmx_not_healthy | draining, restarting or starting; the supervisor is doing planned work | | lmx_stale | no fresh status document, so we cannot see the stack | | lmx_unknown_engine | the engine is not in the document (removed, or hidden from this gateway key) | | lmx_throttled | a 429 on the gateway key. Not retried by the client; err.retryAfterMs says how long the gateway asked for. Every engine on the stack shares the key, so the next endpoint on the same stack will likely be throttled too |

client.test(cfg) reports instead of throwing, and since 0.28.1 its failure result carries the same lmxSkip (and retryAfterMs), so an app that files test results with its breaker can apply the same rule. A health probe can say the same thing: a failing probe result with fault: false makes health.check() return ok: false and record the reason without counting a failure towards the breaker.

Failover across several engines for one job is the app's job; lmx provides none. List endpoints in preference order and take the first one that serves.

Reasoning (reasoning: 'off' | 'full')

How much a model may think is decided by the app, per call, and every provider honours it (since 0.27.0; before that only lmx did). Leave it out, or pass reasoning: 'off', and the model thinks as little as its family allows, so a quick answer stays quick. Pass reasoning: 'full' to let it think:

const r = await ai.complete({
  system, messages, schema,
  reasoning: 'full',
  maxTokens: 4096,        // at least 4096; see the trap below
});                       // and give the endpoint timeoutMs: 120000 or more

When to use it. Use it for judgement tasks where the answer depends on weighing evidence, such as alert triage. Do not use it for quick summaries, rewrites or extraction. They do not get better, only slower.

What it costs. It is much slower, roughly 30–45 s per answer on lab hardware instead of a few seconds, and it spends many more tokens, because the thinking is generated and billed like any other output. Budget for both.

The trap. Thinking comes out of maxTokens. If the limit is too small, the model spends all of it thinking and returns an empty answer, with no error from the server (finish_reason: "length"). The adapter turns that, and a reply cut off inside an unclosed <think> block, into a bad_response AiError saying it ran out of room (and, when the server says so, that the tokens went on internal reasoning), but the work is lost either way. Use maxTokens ≥ 4096 and a timeoutMs ≥ 120 s (120000) on the endpoint, or the deadline cuts the answer off first.

lmx, openai-compatible and lmstudio: what is added to the request body, per model family. The family is read from the model the engine is running (lmx) or the endpoint's model name (others):

| Family | 'off' (the default) | reasoning: 'full' | |---|---|---| | qwen | {"chat_template_kwargs":{"enable_thinking":false}} | {"chat_template_kwargs":{"enable_thinking":true}} | | gpt-oss | {"reasoning_effort":"low"} | {"reasoning_effort":"high"} | | anything else | nothing (lmx logs a warning) | nothing (a warning that 'full' could not be honoured) |

LM Studio ignores chat_template_kwargs (measured live: Qwen3 thought just as long with enable_thinking: false). So on provider: 'lmstudio' a qwen model gets two more switches. One is {"reasoning_effort":"none"} for 'off' and {"reasoning_effort":"high"} for 'full': verified live, but LM Studio documents reasoning_effort only for gpt-oss. The other is Qwen's own documented prompt switch, /no_think for 'off' and /think for 'full', appended to the system prompt (or sent as one when there is none): documented on LM Studio's Qwen3 model pages, but only hybrid Qwen3 reads it. Both are sent, so if a future LM Studio drops one the other still holds. Configure an LM Studio server as lmstudio, not openai-compatible; as openai-compatible it keeps thinking on every call. Plain openai-compatible gets neither, because vLLM validates reasoning_effort and may refuse 'none'.

anthropic: what is sent, per Claude generation. The Messages API differs by generation: Opus 5.5 and Fable cannot turn thinking off (effort is the only lever), Sonnet 5.5 turns it off with between_tools, and several current models refuse temperature, so none is sent to them.

| Models | 'off' (the default) | reasoning: 'full' | temperature | |---|---|---|---| | claude-fable / claude-opus-5-5 | {"output_config":{"effort":"low"}} | {"output_config":{"effort":"high"}} | not sent | | claude-opus-5 | {"output_config":{"effort":"low"}} | {"thinking":{"type":"adaptive"},"output_config":{"effort":"high"}} | not sent | | claude-sonnet-5-5 | {"thinking":{"type":"between_tools"}} | {"thinking":{"type":"adaptive"},"output_config":{"effort":"high"}} | not sent | | claude-sonnet-5 | {"thinking":{"type":"disabled"}} | {"thinking":{"type":"adaptive"},"output_config":{"effort":"high"}} | not sent | | claude-opus-4-7 / 4-8 | nothing | {"thinking":{"type":"adaptive"},"output_config":{"effort":"high"}} | not sent | | claude--4-6 | nothing | {"thinking":{"type":"adaptive"},"output_config":{"effort":"high"}} | sent | | claude--4-5 | nothing | thinking enabled with a budget of maxTokens (at least 1024), added on top of max_tokens, which never exceeds 64000 | sent; not sent with 'full' | | claude-opus-4-1 / claude-opus-4 | nothing | thinking enabled with a budget of maxTokens (at least 1024), added on top of max_tokens, which never exceeds 32000 | sent; not sent with 'full' | | claude-sonnet-4 | nothing | thinking enabled with a budget of maxTokens (at least 1024), added on top of max_tokens, which never exceeds 64000 | sent; not sent with 'full' | | claude-3-7-sonnet | nothing | thinking enabled with a budget of maxTokens (at least 1024), added on top of max_tokens, which never exceeds 64000 | sent; not sent with 'full' | | claude-3 without extended thinking | nothing | nothing (a warning that 'full' could not be honoured) | sent | | anything else | nothing | nothing (a warning that 'full' could not be honoured) | sent |

effort is merged into any output_config the structured-output schema already put there.

On the budget rows max_tokens counts the thinking as well as the answer, so it is capped at the model's output limit (0.27.1). Near the limit the thinking budget shrinks first, down to the API's minimum of 1024, and only then the answer's room: maxTokens: 40000 on Haiku 4.5 sends max_tokens: 64000 with a 24000 budget. Claude 3 models other than 3.7 Sonnet have no extended thinking, so 'full' sends nothing to them and logs a warning; it does not fail the call.

Rules:

• The values are 'off' and 'full'. Leaving the option out (or null) means 'off'. Any other value, for example 'high', true or '', is refused with AiError kind refused before any request is made, whichever provider the endpoint uses.
• The flag is merged after extra, so a raw extra: { chat_template_kwargs: … } or extra: { reasoning_effort: … } cannot change it. Only reasoning can. The option itself is never copied into a request body.
• REASONING_MODES (package root) lists the accepted values for an app that validates its own settings.

Embeddings

Call client.embed(cfg, texts, { signal }) and pass your own resolved embedding config; it never falls back to resolveConfig(). It applies the same policy as complete(): a fast dropped connection is retried once, a 429 fails fast with retryAfterMs on the error, and a config with enabled: false is refused. Embeddings are reported, never limited (see below). The signal aborts the request itself, and a failed answer carries its HTTP status (0.28.2), so a 503 from a model still loading can be told from a broken engine.

On lmx, embed() puts the engine's dimensions on the config you passed as embeddingDimensions (0.28.2; earlier versions promised this and did not do it). Record the width you built an index with and compare it on startup. A model swap changes the width, and an index built at the old width becomes quietly wrong rather than loudly broken.

Embedding usage is reported, never limited (0.29.0). After a successful call, embed() calls budget.recordEmbedding(cfg, { usage, model, ms }, meta) if your budget has that method, where usage is { prompt, completion: 0, total } and model is the model the server says answered. It never calls assertWithinBudget or record, so no ceiling counts or refuses an embedding. Without the hook nothing changes. A hook that throws is logged and the vectors are still returned. A server that sends no usage is not recorded, and neither is an empty input, which makes no request.

Keep these rows out of your completion ledger: ceilings sum every row there. createUsageStore takes the table names, so a second store gives embeddings a ledger of their own, and views/ai/usage-table.ejs can render its byProvider() rows as a separate table.

const embedUsage = createUsageStore({ driver, table: 'ai_embed_usage',
  rollupTable: 'ai_embed_usage_daily', endpointColumn: 'endpoint' });
// budget.recordEmbedding = (cfg, r) => embedUsage.record({ provider: cfg.provider,
//   endpoint: cfg.id, model: r.model, promptTokens: r.usage.prompt, totalTokens: r.usage.total });

Create both tables in a migration from usageSchemaFor(dialect, { endpointColumn: 'endpoint' }), as for ai_usage. The vectors carry the same usage and model as non-enumerable properties, so an app calling the adapter directly can read them too.

Operator screens

• lmxVerify.verify({ instance, statusUrl, statusToken, caCert, enginesKey, engineId?, engineName? }) runs the ordered checks (reachable → certificate → status token → engines key). The engines-key check uses /v1/models, so it costs no tokens and works through the gateway.
• describeStack() and views/ai/lmx-stack.ejs draw the stack panel (active, draining, missing and available engines, and certificate expiry).
• listModelsResult(cfg) returns the engines for the picker, with id, label, role, model facts and the reasoning flag.

Untrusted content

A ticket, an email or a log line is attacker-controlled text. Pass it through untrusted.js or fenced.js, which supply delimited, provenance-tagged data. Use a schema so decoding is constrained. Verify extracted entities against your own records, and keep a person in the loop for anything that changes state.

Conformance

Run lmx's examples/client-smoke.mjs against the stack before blaming this package. Exit code 0 means the contract holds, so a failure is on this side.