npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@dapplelight/signet-ai

v0.5.0

Published

Provider-agnostic structured LLM calls with budget caps, caching and failover.

Readme

@dapplelight/signet-ai

Provider-agnostic structured LLM calls with budget caps, caching and failover. Resolves signet#1.

The extractor is structured output only. Every extract call goes through a Zod schema and returns a typed, validated object, never free text. Tool loops, streaming chat, embeddings and grounded answers live on their own subpaths (see "AI SDK helpers" below), so the core stays small and needs no ai install.

import { createExtractor, memoryCache, createBudgetLedger } from "@dapplelight/signet-ai"
// The AI SDK adapter is a SUBPATH, not on the barrel — see "Why the adapter is
// a separate import" below. Requires `ai` 7 or later.
import { aiSdkProvider } from "@dapplelight/signet-ai/providers/ai-sdk"
import { createWorkersAI } from "workers-ai-provider"

const workersAi = createWorkersAI({ binding: env.AI })

const ai = createExtractor({
  providers: [
    aiSdkProvider({ name: "workers-ai", model: workersAi("@cf/meta/llama-3.3-70b-instruct-fp8-fast"),
      pricing: { perCallUsd: 0.0005 } }),  // an estimate: Workers AI bills in Neurons
    aiSdkProvider({ name: "backup", model: someOtherModel,
      pricing: { inputPerMTokens: 0.15, outputPerMTokens: 0.6 } }),
  ],
  cache: memoryCache(),
  // $5/day, fails closed. Memory is per isolate: see "Budgets across isolates".
  ledger: createBudgetLedger(5, 86_400),
})

const { value, meta } = await ai.extract({
  schema: intentSchema,
  input: "8000 se kam ka snowboard",
  instructions: "Pull the budget, size and category from this shopping request.",
  budget: { maxCostUsd: 0.002 },
})
// meta: { provider, model, usage, costUsd, cacheHit, attempts, durationMs }

Model choice matters more than anything in this package

Benchmarked through this extractor on real inputs by the first production consumer, five free Workers AI models, same work:

| Model | Latency | |---|---| | @cf/meta/llama-3.3-70b-instruct-fp8-fast | 1.1s | | @cf/zai-org/glm-4.7-flash | 17.7s / 30.0s / 21.0s, one ALL_PROVIDERS_FAILED | | @cf/meta/llama-3.1-8b-instruct | fails instantly — no structured output on this path |

glm-4.7-flash was this README's example and is unusable inline. If you copied it before, change it. End-to-end latency in production went from 17–30s to 1.6–3.0s including a Shopify call.

A model with no structured-output support fails closed in ~0.1s, which is the correct behaviour — but a provider array containing only that model will always fail.

Schema validity is not truth

This package guarantees shape, not correctness. "Typed and validated or it is an error" is easy to read as "correct". It is not the same claim, and the difference matters.

Observed in production, all schema-valid and all wrong:

  • maxPrice: 8000 for "snowboards for a beginner" — no price in the input
  • a field meant to hold "ski jacket" returned "The user asked for a warm skiing item with a maximum price of $900…"
  • every model returned inStock: true for requests that never mentioned availability

Re-validation passes all of these, correctly — they are the right shape with invented content. If a hallucinated value would cause harm, verify each claim against the input text and discard what the text does not support. That layer is application-specific and deliberately not in this package.

Decisions that are load-bearing

Providers are an array, tried in order. Adding or reordering is config, not code. Exhausting Workers AI's free allocation (error 3036, delivered as a 429) fails over to the next entry with no product change.

Budget fails closed. Once a window is exhausted, calls are refused. A fallback value does not mask a budget refusal — running out of money is a configuration problem the caller must see, not a soft failure.

A ledger needs prices. With a ledger set, createExtractor throws AI_UNPRICED_PROVIDER if any provider has no pricing, because an unpriced provider counts as $0 and the cap never fills. Use perCallUsd for flat-rate providers, or pass allowUnpriced: true when one really is free.

Only transient errors retry. A schema violation or a bad key fails identically every time; retrying spends budget and latency for a guaranteed failure. Unknown error shapes are treated as permanent — "retry on doubt" is how a cap gets spent on something that was never going to work.

The SDK's own retries are disabled (maxRetries: 0). Retrying in both places multiplies spend silently underneath a cap.

Output is re-validated against the schema even though the provider claims to have done it. A provider returning an unvalidated object is the exact failure this package exists to prevent.

The schema and the model are part of the cache key. The same prompt against a changed schema is a different question; reusing the old answer returns data that no longer validates. Switching models should not keep serving the old one's answers either, so the key also covers the provider chain (name/model of each provider, in order).

Persistent caches

memoryCache() lives and dies with one isolate. For answers that survive a restart and are shared everywhere, pass a persistent Cache; it is the same two-method interface (get, set(key, entry, ttlSeconds)), so your own store works too.

// Workers KV: the namespace binding as is (or any KeyValueLike, e.g. upstashKv)
createExtractor({ providers, cache: kvCache(env.AI_CACHE), cacheTtlSeconds: 86_400 })

// A table in your database (SQLite, D1 or Postgres), from @dapplelight/signet-db
import { dbCache } from "@dapplelight/signet-db"
createExtractor({ providers, cache: dbCache(db, schema.aiCache) })

Entries carry their own expiry, checked on read, so a store that keeps a key longer (Workers KV's 60-second minimum) still expires it on time. A cache that is down reads as a miss and a failed write is dropped (both emit ai.cache_error to onEvent): an answer already paid for is never thrown away. Each hit is reported to onUsage with cacheHit: true, costUsd: 0 and savedUsd, what the original call cost.

Budgets across isolates

The default ledger keeps spend in memory, in one isolate. On Workers each isolate has its own and a cold start empties it, so "$5/day" really means $5 per isolate per cold start. Give the ledger a shared store:

// Exact: one Durable Object per budget key. Free-plan Workers include Durable Objects.
import { BudgetCounter, durableObjectBudgetStore } from "@dapplelight/signet-ai/budget-do"
export { BudgetCounter } // from the worker entry, so the binding finds the class

const ledger = createBudgetLedger(5, 86_400, {
  store: durableObjectBudgetStore(env.AI_BUDGET),
  perKeyUsd: 0.5, // each user gets $0.50/day within the $5
})
await ai.extract({ ...args, budget: { key: userId } })
// wrangler.jsonc
"durable_objects": { "bindings": [{ "name": "AI_BUDGET", "class_name": "BudgetCounter" }] },
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["BudgetCounter"] }]

kvBudgetStore(env.KV) is the cheap alternative, and it is approximate: its window is fixed (it resets on the boundary) and concurrent writes can lose a record. Any object with spent and record is a BudgetStore, for Redis or a database off Workers.

Calls in flight are not reserved, so concurrent calls can overshoot a cap by their own cost.

Usage reporting

Every call can report its tokens and cost. Pass onUsage to createExtractor or to any helper below; each call that reached a model reports once:

import { analyticsUsage } from "@dapplelight/signet-ai"

const ai = createExtractor({
  providers,
  onUsage: (r) => log.info("ai.usage", r),
  // or straight to an AnalyticsSink from @dapplelight/signet-server/analytics:
  // onUsage: analyticsUsage(sink),
})

A report is { operation, provider, model, usage, costUsd, key, durationMs }. operation is extract, tools, stream, embed or ground; key is the budget key (a user id), so cost can be attributed per user. A cache hit is reported with cacheHit: true, costUsd: 0 and savedUsd (the original call's cost), so the cache's savings add up beside the spend. Fallbacks spend nothing and are not reported. A call refused by the per-call cap after the spend is reported, because the money is gone. A hook that throws, or a sink that rejects, is ignored: analytics must not fail a paid call. analyticsUsage(sink, name?) writes one ai.usage event per report, with cacheHit: 1 and savedUsd on a hit.

AI SDK helpers

Requires ai 7 or later. Each helper takes a model from any AI SDK provider package, so nothing here names a provider. Each takes onUsage, pricing, key, provider (the label in reports) and abortSignal, and turns the SDK's own retries off, as the extractor does.

Tool loops

import { runTools } from "@dapplelight/signet-ai/chat"
import { tool } from "ai"

const r = await runTools({
  model,
  tools: { lookupOrder: tool({ description, inputSchema, execute }) },
  prompt: "Where is order A1?",
  maxSteps: 5, // default 5; each model call is a step
  onUsage,
})
if (r.capped) { /* stopped at maxSteps while the model still wanted tools */ }
// { text, steps, capped, toolCalls, usage, costUsd }

usage is the total over every step. A perCallUsd price is charged once per step. Check capped: a capped run's text is often empty.

Streaming chat

The server returns streamChat(...); the client reads it with useSignetChat, which wraps useChat from @ai-sdk/react (an optional peer, install it in the web app).

// server (Hono, Workers, Next route handler: anything that returns a Response)
import { streamChat } from "@dapplelight/signet-ai/chat"

app.post("/api/chat", async (c) => {
  const { messages } = await c.req.json()
  return streamChat({ model, messages, tools, maxSteps: 5, onUsage, key: userId,
    abortSignal: c.req.raw.signal })
})
// client
import { useSignetChat } from "@dapplelight/signet-ai/react"

const { messages, sendMessage, status, stop, usage } = useSignetChat({ api: "/api/chat" })

Usage reaches onUsage on the server when the stream ends, and the client as message.metadata.usage on each assistant message; usage from the hook is the conversation's sum. api and headers are read once per chat id, as in useChat; pass headers as a function for a value that changes. Authenticate the route and cap messages yourself: the client sends the whole history.

messages is untrusted input. streamChat drops system messages (set instructions with system) and every tool part from the history, so a browser cannot plant a system prompt or a fake tool result. Tools the model calls during the request still run; their results from earlier turns are not replayed.

Embeddings and search

import { embedDocuments, searchDocuments } from "@dapplelight/signet-ai/ground"

const indexed = await embedDocuments(embeddingModel, docs) // [{ id, text, embedding }]
// store `indexed` anywhere (JSON column, KV); vectors are plain number arrays
const hits = await searchDocuments(embeddingModel, "how long is delivery", indexed, {
  topK: 5, minScore: 0.3,
})
// [{ id, text, score }], best first

There is no vector store: ranking is cosine similarity in memory, which is fine for a few thousand documents. Past that, use the platform's index (Vectorize, pgvector) and pass its results to groundedAnswer.

Grounded answers

import { groundedAnswer } from "@dapplelight/signet-ai/ground"

const r = await groundedAnswer({ model, question, documents: hits })
if (!r.grounded) return "I couldn't find that in our help pages."
// r.answer, r.citations: [{ id, quote }], r.rejected

The model must cite each claim with a document id and a verbatim quote. Every citation is checked: an unknown id, or a quote that does not appear in that document (ignoring case and whitespace), goes to rejected. grounded is true when at least one citation survives. This checks the citations, not the reasoning: a model can quote a real line and still over-claim, so show the quotes beside the answer. Documents are wrapped in <document> tags and the model is told their text is data, which reduces but does not remove prompt injection from document content.

Testing the helpers

Use the SDK's mock models from ai/test (MockLanguageModelV4, MockEmbeddingModelV4); test/sdk.ts in this package shows each helper driven by them with no network.

Why the adapter is a separate import

aiSdkProvider lives at @dapplelight/signet-ai/providers/ai-sdk and is deliberately not re-exported from the barrel.

export * resolves eagerly in ESM, so barrelling it would make importing createExtractor alone pull in ai — turning an optional peer into a hard requirement. Anyone using only fakeProvider in tests, or writing their own Provider, would need a dependency they never asked for, and would only find out at runtime.

The core package is transport-agnostic and depends on nothing but @dapplelight/signet-lib. Same reasoning as @dapplelight/signet-server/http needing hono only if you use it.

import { createExtractor } from "@dapplelight/signet-ai"              // no `ai` needed
import { aiSdkProvider } from "@dapplelight/signet-ai/providers/ai-sdk" // needs `ai`
import { runTools, streamChat } from "@dapplelight/signet-ai/chat"     // needs `ai`
import { groundedAnswer } from "@dapplelight/signet-ai/ground"         // needs `ai`
import { useSignetChat } from "@dapplelight/signet-ai/react"           // needs `ai`, `@ai-sdk/react`

Testing without network or spend

import { createExtractor, fakeProvider } from "@dapplelight/signet-ai"

const ai = createExtractor({ providers: [fakeProvider({ budget: 8000, category: "snowboard" })] })

fakeProvider takes a fixed value, a list, or a function of the input, and can be told to fail N times first to exercise retry and failover paths.

Errors

AiError extends ApiError from @dapplelight/signet-lib/api, so an AI failure is handled like any other API failure. Codes: AI_BUDGET_EXCEEDED (429), AI_ALL_PROVIDERS_FAILED (502), AI_SCHEMA_INVALID (502), AI_NO_PROVIDERS (500).

Jev: @dapplelight/signet-ai/jev

A safe wrapper for Jev, TypeSafe's System One classifier: text in, typed decision out. app-a, app-b and app-d each had their own copy with the same model pin, 0.95 floor and 8 s timeout. One of them had no floor and threw.

import { createJev } from "@dapplelight/signet-ai/jev"

// Build it where env exists (a handler, a CLI main), never at module scope.
const jev = createJev({
  apiKey: env.JEV_API_KEY,        // missing: every call returns its fallback
  // model = "jev-1.13.0", minConfidence = 0.95, timeoutMs = 8000
  // allowSensitive = false       // the explicit opt-in for private/financial text
  onEvent: (e) => log.info("jev", e),   // redacted: no key, state or question text
})

const d = await jev.choice({
  question: "Which spending category fits this bank transaction?",
  options: { food: "Restaurants, delivery", travel: "Flights, cabs", unsure: null },
  state: `Money paid out. Narration: "${narration}"`,
  fallback: "unsure",
  sensitive: true,                // required: bank text is financial
})
// { value, confident, source: "jev" | "fallback", confidence?, reason?, answer? }
if (d.confident) apply(d.value)
else review.push({ best: d.answer, confidence: d.confidence })

| Method | Jev primitive | value | |---|---|---| | choice({ question, options, fallback, ... }) | choice, up to 255 labels, as a list or label -> description | one of the labels | | score({ question, levels, fallback, ... }) | score, 2-10 ordered levels | a number, can fall between levels | | noul({ question, fallback, ... }) | noul | a boolean; probability carries the raw yes-probability | | batch({ state, questions, ... }) | several named questions about one state, in one request | a decision per name |

Every call takes state, sensitive, fallback, and optionally minConfidence and signal. jev.enabled is false without a key, so an app can skip preparing work it will not send.

It never throws for the failures that matter. Every call resolves to a decision the app can act on. The fallback is used, with a reason, when:

| reason | When | |---|---| | sensitive_not_allowed | sensitive: true and the instance lacks allowSensitive: true. Checked first; fetch is never called | | no_key | apiKey missing or blank. Fetch is never called | | low_confidence | below the floor. answer and confidence keep what Jev said, for a review queue | | timeout | over timeoutMs, even if the fetch ignores its abort signal | | http_error | a non-2xx response, such as 401 or 429. The event carries status | | network_error | fetch rejected | | invalid_response | not JSON, a missing answer, or a label outside the offered set | | invalid_request | no questions, an out-of-range floor, score levels outside 2-10, or a fallback not among the options | | aborted | the caller's signal fired |

Decisions that are load-bearing:

  • The 0.95 floor is on by default. On app-a's first real run, 0.85-0.94 let wrong guesses through and 0.95 let none through. Pass minConfidence per call where a wrong answer is cheap, as app-b's loop does.
  • sensitive is required by the type, and a JS caller that omits it is treated as sensitive. The opt-in belongs to the instance, so the owner decides it once in config, not each caller in code. Deciding what is safe to send is still the app's job: app-a strips digit runs from narrations.
  • The noul confidence is max(p, 1 - p). Jev reports none for noul, so a 0.7 "yes" falls back at the default floor. Rank by probability if you need to.
  • Fetch only, no SDK. The API is a single POST /v1/systemone with Bearer auth. @typesafe-ai/sdk retries twice by default with a per-attempt timeout, so its "8 s" could reach about 27 s. It also falls back to TYPESAFE_API_KEY from env and logs request bodies at debug level. Here timeoutMs is the total, with no retries, because the fallback is the retry.
  • The key goes into the Authorization header and nowhere else. Upstream error messages and response bodies are never copied into a result or event, because a transport can echo headers. A test drives every failure path through a transport that tries to leak the key.
  • Not on the barrel. Import it from @dapplelight/signet-ai/jev. It has no dependencies.

Migrating the existing copies

  • app-a (packages/mcp/src/lib/jev-categorize.ts): drop JEV_MODEL, TIMEOUT_MS, makeJevClassifier, the TypeSafeClient import and the @typesafe-ai/sdk dependency. Build createJev({ apiKey: env.JEV_API_KEY, allowSensitive: env.JEV_ALLOW_PRIVATE_MODE === "1" }), then call jev.choice({ ..., fallback: "unsure", sensitive: Boolean(crypto) }). That replaces assertPrivateModeAllowed inside jevCategorize, and a try/catch counting errors becomes reason not being low_confidence. The confidence comparison moves into the wrapper; low_confidence rows go to review using answer and confidence. Keep sanitizeNarration, the unsure label and the no-auto-apply rule for transfers: those are product rules.
  • app-b (scripts/jev-client.mjs): delete the file. decide() in loop/jev-steps.mjs becomes jev.choice({ ..., fallback: safe, minConfidence, sensitive: false }). Behaviour fix: jev-pr-risk.mjs, jev-triage.mjs and jev-stop-gate.mjs call classify with no floor, so a 0.3 guess labelled a PR, and any error or missing key threw and killed the script. They now get the 0.95 floor and a fallback: medium for PR risk, structural for the stop gate so E2E still runs, and no tier for triage. assessCompleteness becomes jev.noul({ ..., fallback: false }).
  • app-d (src/lib/jev.ts): delete JEV_MODEL, MIN_CONFIDENCE, TIMEOUT_MS, getClient, and the try/catch returning null in jevClassify. jevEnabled() becomes jev.enabled. The null-or-low-confidence check in match.ts confident() and in route.ts becomes d.confident. jevScore becomes a jev.batch of noul questions per chunk of 25, reading probability with a fallback of true so an unscored video is kept. All app-d calls send public content, so they use sensitive: false.

Out of scope

Chat history storage, vector databases, and agents beyond a single tool loop. Add when a second product needs them, not before.

Test

bun run test runs 62 extractor assertions (failover, retry classification, budget fail-closed, cache keying, schema enforcement), 59 Jev assertions (the confident path, every fallback reason, the privacy gate, batch, and the key never leaking) and 37 AI SDK helper assertions (the adapter, onUsage, tool loops and the step cap, streaming, embeddings, search and citation checks), all with no network.