npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@shubhamdeol/locale-ota-ai

v1.0.0

Published

Prompt builder, the generateText call, provider registry, usage capture.

Readme

@shubhamdeol/locale-ota-ai

The translation engine: the provider registry, the prompt builder, the batch call, the validator and the job runner. It knows nothing about HTTP and nothing about Postgres. It talks to a JobStore the server implements, so the same engine runs in a route, in a worker or in a test.

Status: phase 3. Version 0.0.0, unpublished.

Install

bun add @shubhamdeol/locale-ota-ai

Providers

resolveModel(spec) turns a spec string into an AI SDK LanguageModel. No model id is hardcoded in this package, so a new model needs no release.

| Spec | Factory | API key variable | | --- | --- | --- | | google:<model-id> | google() from @ai-sdk/google | GOOGLE_GENERATIVE_AI_API_KEY | | openai:<model-id> | openai() from @ai-sdk/openai | OPENAI_API_KEY, and OPENAI_BASE_URL when set | | a LanguageModel object | passes through | whatever that provider reads | | anything else | — | throws AiError with code unknown_provider |

Both variables were read out of the installed packages, @ai-sdk/[email protected] and @ai-sdk/[email protected]. Neither provider reads its key when the model is built, only when a request is sent, so resolveModel never throws for a missing key and a dashboard can show the model choice without credentials. PROVIDER_KEY_VARIABLES exports the same table.

The public API

| Export | What it does | | --- | --- | | resolveModel(spec) | The spec table above. describeModel(spec) names a model for jobs.model. | | buildPrompt({ app, language, format, items }) | { system, prompt } for one batch. | | translateBatch({ model, app, language, format, items, signal? }) | One model call: { results, usage, raw }. | | validateResults({ items, results, format }) | { accepted, rejected } through core validateTranslation. | | buildFixItems(rejected, items) | The items to ask again, each quoting its errors. | | planWork({ releaseKeys, sourceValues, existing, language, only? }) | The work items one language is missing. | | runJob(options) | The whole job: plan, batch, call, validate, fix, write, resume. | | memoryJobStore() | A JobStore in memory, for tests and as the reference. | | AiError | The one error type: a code string and a details object. |

Every item is either plain or plural:

{ kind: "plain",  namespace, key, sourceHash, source, context? }
{ kind: "plural", namespace, key, sourceHash, source, context?,
  ordinal, forms: { one, other }, needed: ["zero", ...], sourceHashes: { zero, ... } }

sourceValues is a flat map keyed by sourceKey(namespace, key), which joins the two with a NUL byte. existing is the list of { namespace, key, sourceHash } the language already holds, of any origin, which is why a human override is never re-asked: it is simply a row.

How a batch flows

planWork  ->  batchWork  ->  claimBatch  ->  translateBatch  ->  validateResults
                                                   |                   |
                                                   |            rejected? buildFixItems
                                                   |                   |
                                                   |            translateBatch again
                                                   v                   v
                                        insertTranslations      insertFlags
                                                   |
                                             markBatch done
                                                   |
                                    updateJobCounts + onProgress
  1. Plan. planWork asks for every release key with no row at the same source hash, and for every plural group that still misses a target form. A changed English value has a new hash, so it is asked again; an unchanged one never is.

    only, on planWork and on runJob, narrows the plan to the keys it names and asks for them whatever rows they already hold. That is the dashboard's "retranslate one key": without it a key with a row, human or not, is never asked again, which is the point of existing. A plural group is named by any of its form keys or by its base, and the whole group is then asked for every category the language needs. The written row is a new append-only row at a higher revision, so it supersedes the one that stood, a human override included: naming a key is an explicit instruction to replace it.

  2. Batch. Core batchWork splits the list at 150 items or 8 KB of English text, in order, so the same input always splits the same way. One job_batches row per batch.

  3. Claim. Each of concurrency workers takes one pending batch at a time through store.claimBatch(jobId), which must be atomic in the store: in Postgres that is one UPDATE ... WHERE id IN (SELECT ... FOR UPDATE SKIP LOCKED) RETURNING *.

  4. Call. One generateText call with output: Output.object({ schema }), where the schema is a zod object built for this batch: every item id is a required property, a string for a plain key and an object of the needed plural categories for a plural base. The item id is namespace:key, plus #ordinal for an ordinal group, so the model answers under keys it can read. Usage comes back as inputTokens and outputTokens.

  5. Validate. Core validateTranslation checks every value and every plural form: markers as a set, HTML tags as a multiset, no empty value. A plural item is all or nothing, so it never lands half translated.

  6. Fix. One more call for whatever was rejected, each item carrying a context line that quotes the errors. Anything still rejected gets a flags row and no translation row, so the client falls back to English for that key.

  7. Write. insertTranslations with origin ai, the job id and the batch id. The revision is assigned by the store, because it is the next value of a per-app counter and only the database can hand it out without a race.

  8. Report. updateJobCounts and onProgress after every batch.

Retries are the runner's, not the SDK's: translateBatch passes maxRetries: 0 so that one batch attempt is one request. A retryable error waits 1 s, then 4 s, then 16 s, each with half jitter, up to retries attempts; after that the batch is failed and the other batches carry on. An error counts as retryable when the SDK's APICallError.isRetryable says so, or the status is 408, 409, 429 or 5xx, or an APICallError carries no status at all, which is a network error. A schema mismatch is not retryable: if the model cannot fill the shape it was given, the batch fails and the dashboard shows why.

A job with any failed batch ends failed, otherwise done.

Counts

keysTotal, keysDone and keysFlagged are in work items, counted across every language. A plural base is one item however many forms it produces. With no failed batch, keysDone + keysFlagged === keysTotal; a failed batch's items are neither, because nothing was written and nothing was flagged for them.

Resume and cancel

The job is a row and every batch is a row, so a run is resumable.

  • A run with no jobId calls findOpenJob({ appId, releaseId, languages }). An open job, pending or running, is picked up: its batches already exist and are not planned again.
  • On start, resetStaleBatches(jobId, staleAfterMs) puts running batches older than ten minutes back to pending, which is how the batches of a killed process come back.
  • The signal is checked before every claim and handed to every request. A run stopped by its signal leaves the job running and releases the batch it was holding back to pending, so the next run finishes the job rather than failing it.

The JobStore

interface JobStore {
  createJob(input: CreateJobInput): Promise<Job>;
  getJob(id: string): Promise<Job | null>;
  findOpenJob(query: FindOpenJobQuery): Promise<Job | null>;
  createBatches(jobId: string, batches: readonly NewJobBatch[]): Promise<JobBatch[]>;
  listBatches(jobId: string): Promise<JobBatch[]>;
  claimBatch(jobId: string): Promise<JobBatch | null>;
  markBatch(id: string, status: BatchStatus, options?: MarkBatchOptions): Promise<void>;
  resetStaleBatches(jobId: string, staleAfterMs: number): Promise<number>;
  insertTranslations(rows: readonly InsertTranslationRow[]): Promise<void>;
  insertFlags(flags: readonly Flag[]): Promise<void>;
  updateJobCounts(jobId: string, counts: JobCounts): Promise<void>;
  finishJob(jobId: string, status: JobStatus): Promise<void>;
}

Job, JobBatch and Flag are the columns of plan section 4 in camel case. JobCounts is a partial of the five jobs counters. InsertTranslationRow is a core TranslationRow without its revision, plus appId, jobId and batchId.

Rules the store must keep:

  • claimBatch is atomic. Two workers, or two processes, never get the same batch.
  • createJob honours the status the caller asks for. The runner asks for running.
  • insertTranslations assigns one revision per call, the next value of the app's counter.
  • markBatch(id, "pending") releases a claim: clear started_at, keep attempts.
  • listBatches is read-only. The dashboard needs it anyway for per-batch errors and tokens.

Plurals

Core pluralWork gives the English forms a release ships and the target forms a language needs, from Intl.PluralRules. An English pair becomes six keys for Arabic and one for Japanese. The engine writes one row per generated category, keyed with core pluralKey, so entry_zero, entry_one and the rest.

A generated row is filed under the source hash of the English form it was translated from: its own category when the release ships it, otherwise the other form. That keeps the overlapping categories, one and other for most languages, composable against the release key set without any change to core compose.

Tests

bun test packages/ai     # from the repository root
bun run typecheck        # from this folder
bun run build            # from this folder

Every test uses MockLanguageModelV4 from ai/test. Nothing in this package's tests reaches a provider. src/fake-model.ts is the mock: it reads the batch out of the prompt and answers [<language>] <source> for every key, keeping every marker and tag, filling every needed plural category, and it can be told to drop a marker once, to drop one every time, or to throw a retryable error. It is test support and is not exported from the package entry.

The runner tests take their 2,000 key release from the core fixture generator at packages/core/test/fixtures/generate.ts.