@speles7172/ai-client

AI features over AWS Bedrock. One idea: a feature.

Every AI flow an application offers — correcting a draft, reading a photographed receipt, translating a message, and whatever it adds next — is an entry in a registry with a key, a prompt and a model. Which model runs a flow is a setting an administrator picks from the whole live Bedrock catalog, not a code path. And every call is tagged with the project, the environment and the feature, so the bill can be read.

npm install @speles7172/ai-client

Requires Node 22+. Two entry points: . (the Bedrock runner and the model catalog) and ./core (everything else — dependency-free, and what @speles7172/ai-console imports).

Using it

import {
  createAiService,
  createBedrockRunner,
  configModelSource,
} from '@speles7172/ai-client';

const ai = createAiService({
  runner: createBedrockRunner({ region: 'us-east-1' }),
  attribution: { project: 'peles-utils', environment: process.env.STAGE ?? 'dev' },
  models: configModelSource((key) => settings.text(key, null)),
});

// Grammar and spelling, in a tone the writer picks.
const { text } = await ai.correctGrammar({ text: draft, tone: 'friendly', context: thread });

// Translation.
await ai.translate({ text: draft, targetLanguage: 'Hebrew' });

// A picture, a scan or a PDF: what it says, and the fields you asked for.
await ai.scan({
  attachments: [{ contentType: 'image/jpeg', data: base64, filename: 'receipt.jpg' }],
  fields: [
    { name: 'total', type: 'number' },
    { name: 'paid_on', type: 'date' },
    { name: 'vendor', type: 'text', description: 'the payee on the receipt' },
  ],
});

The three flows, and the fourth

grammar, image_scan and translate are built in. Anything else is one entry:

import { createAiService, defineAiFeatures } from '@speles7172/ai-client';

const features = defineAiFeatures([
  // Re-point a built-in at a better model. Everything not mentioned survives.
  { key: 'grammar', defaultModel: 'us.anthropic.claude-3-5-haiku-20241022-v1:0' },
  // Add one of your own.
  { key: 'summarize', label: 'Thread summary', maxTokens: 512 },
]);

const ai = createAiService({ runner, attribution, features });

await ai.generate({ feature: 'summarize', prompt: `Summarise:\n${thread}` });

A declared feature gets a configuration key (AI_MODEL_SUMMARIZE), a picker on the settings page, and a cost tag — none of which anyone has to wire up separately.

Choosing the model in configuration

aiConfigDefinitions() hands @speles7172/config-client the declarations, so the settings page has one picker per flow and nobody types the keys twice:

import { defineConfig, createConfigStore } from '@speles7172/config-client';
import { aiConfigDefinitions } from '@speles7172/ai-client/core';

const config = createConfigStore({
  execute: pool.query.bind(pool),
  schema: defineConfig([...appSettings, ...aiConfigDefinitions(features)]),
});

const settings = await config.snapshot();
const ai = createAiService({
  runner,
  attribution,
  features,
  models: configModelSource((key) => settings.text(key, null)),
});

The two packages do not depend on each other. aiConfigDefinitions returns objects that are structurally ConfigDefinitions, and TypeScript is structural — the same trade the Executor types make across this repository.

Answering null means "no override", so a setting that has never been written is indistinguishable from one set to the declared default. That is what lets a default change in a release and take effect.

Showing every model

import { listBedrockModels } from '@speles7172/ai-client';

const models = await listBedrockModels({ region: 'us-east-1' });

Asked of Bedrock rather than carried here: AWS adds models continuously and access is granted per account, so a written catalog is wrong within the month — in the direction of hiding models the account has just been granted. Two calls are unioned: the foundation models invocable on demand, and the cross-region inference profiles, which is where the newest models live and the only way they can be invoked at all.

Needs bedrock:ListFoundationModels and bedrock:ListInferenceProfiles. Without either it returns a shorter list rather than failing — and a model id typed into the picker works regardless.

Cost tracking

Two mechanisms, easy to confuse because both are called tagging:

• Request metadata — the project / environment / feature pairs this package puts on every single Converse call. They land in Bedrock's model-invocation logs, which is where "how many calls did the grammar feature make in staging last month" is answered. Free, and needs no infrastructure.
• An application inference profile — an AWS resource your application creates, carrying real cost-allocation tags. Invoking its ARN instead of a bare model id is the only thing that makes Bedrock usage appear under a project in Cost Explorer.

This package does both halves it can: it stamps the metadata, and it invokes whatever model id it is given — so pointing a feature at an application inference profile ARN is a configuration change, not a code change. Creating that profile is yours, the way the tables are in the sibling packages.

An attribution with no project or no environment is refused at createAiService, in your deploy, rather than per request. An untagged invocation cannot be attributed afterwards; there is nothing left to attribute.

Running where Bedrock cannot be reached

createAiService lives in ./core and takes an injected runner, so a handler in a VPC with no route to Bedrock runs the same flows through a bridge Lambda — with the same prompts, the same parsing and the same tags, and without the AWS SDK in its bundle:

import { createAiService } from '@speles7172/ai-client/core';

const runner = async (request) => {
  const answer = await invokeBridge({ action: 'generate', request });
  return answer;
};

What it does not do

• No infrastructure. No Lambda, no inference profile, no IAM. Your application grants bedrock:InvokeModel over the model and profile ARNs it wants to allow.
• No access control. Who may use a feature is your endpoint's decision. The same stance @speles7172/audit-client and @speles7172/file-client take: a permissive default that looks like a permission system is worse than an obvious absence of one.
• No streaming. Every flow here is one prompt and one answer. A chat UI wants ConverseStream and a different shape of API; adding it to these helpers would make the simple case pay for the complicated one.
• No conversation state. Nothing is stored. A flow that needs the thread passes it as context.

Traps

Converse, not InvokeModel. One request and response shape across every model family, which is the whole reason a model can be a setting. InvokeModel takes each family's own body — Llama wants {prompt} and answers {generation}, Claude wants {messages} and answers {content}, Nova wants {schemaVersion} — so switching a feature's model under it can only ever mean switching within one family.

A profile-only model is retried once, through the region's profile. Newer models are not invocable by their bare id at all, and Bedrock's refusal reads as though the model does not exist. Without the retry, picking one from the live catalog is a support ticket.

A document's name is validated by Bedrock, strictly. Alphanumerics, whitespace, hyphens, parentheses and brackets — and no consecutive whitespace. Invoice #4471 (final).pdf is rejected outright, and the error names the document field rather than the filename. toAttachment sanitises it, so one place gets it right.

The picture flow needs a model that can see. image_scan is declared vision; a text-only model configured for it fails at the model, not here. The catalog reports which models take an IMAGE input, because nothing about the ids says so.

A model answers prose unless something stops it, and sometimes anyway. sanitizeText removes the code fence, the Here is the improved draft: label and the wrapping quotes; extractJson matches braces rather than pattern matching, because the common failure is valid JSON with a sentence in front of it — and the next one is a sentence after it containing a brace.

Extraction runs at temperature 0, deliberately. A field read off an invoice at 0.7 is occasionally a field that was never on it, and a plausible invented value is worse than a missing one.

An unknown tone throws. A silent fallback to neutral would mean a typo in a stored setting turns "always formal" into "never formal", and the only symptom is prose that reads slightly wrong to whoever set it. The service's defaultTone is typed for the same reason — narrow a stored value with isAiTone before passing it.

Licence

MIT.