npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

typesafe-gates

v0.2.1

Published

Probabilistic decision gates for LLM pipelines on TypeSafe AI's System One (Jev): typed Score, Choice and yes/no judgments with explicit thresholds, a stated fail-open or fail-closed policy per gate, all-or-nothing batches, time budgets, and usage account

Readme

typesafe-gates

Decision gates for LLM pipelines, built on TypeSafe AI's System One (Jev). A gate asks Jev one or more typed questions about each item (a Score on a fixed rubric, a Choice from fixed labels, or a yes/no), gets back a probability table instead of free text, and lets your code own the keep rule, the threshold, and what happens when the judge is unavailable.

import { createJudge, verdictsOrNull } from "typesafe-gates";
import { relevanceGate, relevanceMinProbability } from "typesafe-gates/gates";

const judge = createJudge({ recordUsage: (r) => db.insert(aiUsage).values(r) });

const result = await relevanceGate.run(judge, items, {
  name: "Example Band",
  about: "a four-piece indie rock band from Pittsburgh",
  kind: "entity",
});

const probabilities = verdictsOrNull(result); // null when Jev was unavailable
const kept = probabilities
  ? items.filter((_, i) => probabilities[i]! >= relevanceMinProbability())
  : items; // fail-open: keep the previous path for the whole batch

Every gate in this package came out of a production newsletter platform where "ask Claude to return the indices to keep" had three failure modes: the parse could fail (a whole batch silently dropped), the model could soft-refuse (a polite decline parses as nothing), and there was no signal for how relevant an item was, so a threshold was prompt wording instead of a number.

The long version of where these gates came from, with the audit numbers behind each threshold, is in docs/seven-production-gates.md.

Install

npm install typesafe-gates @typesafe-ai/sdk

@typesafe-ai/sdk is a peer dependency (0.6 or later). Set TYPESAFE_API_KEY, or pass your own client to createJudge.

What a gate is

import { noul } from "@typesafe-ai/sdk";
import { defineGate } from "typesafe-gates";

export const spamGate = defineGate({
  name: "spam_screen",
  policy: "fail-closed",
  questions: {
    spam: noul("Is `message` unsolicited bulk mail to `inbox`?", {
      true: "bulk, automated, or scam mail",
      false: "written by a person to this inbox",
    }),
  },
  state: (message: string, inbox: string) => ({ message: message.slice(0, 4000), inbox }),
  interpret: (answers) => answers.spam.noul, // P(spam)
});

A gate declares four things:

  • questions: SDK question builders (score, choice, noul). Several independent questions over the same state ride in one request.
  • state: the evidence for one item. Keep it small and explicit. A bare name makes a model guess what the subject is; a 50 KB thread bills in full for a judgment that needed its first 6 KB.
  • interpret: a pure function from one answer set to the verdict your code uses. This is where "P(level ≥ 2)" or "P(ad_inquiry) + P(negotiation)" lives.
  • policy: fail-open or fail-closed. Declared, not inferred, because the two cases have opposite safe defaults.

gate.run(judge, inputs, context) returns either { status: "judged", verdicts, model, usage, policy } or { status: "unavailable", reason, error?, usage, policy } with reason one of no_client, request_failed, over_budget, aborted. Three helpers cover the common shapes: verdictsOrNull (fail-open), verdictOrNull (one item), verdictsOr(result, reason => fallback) (fail-closed).

The policy covers the judge being unavailable, not bugs in your gate. If state or interpret throws (for example on a malformed webhook payload), run rejects with that error. A fail-closed caller should catch it and take the same conservative path:

const decision = await inboundScreenGate
  .runOne(judge, email, inbox)
  .then(inboundDecision, () => ({ action: "hold", reason: "screen_error", alert: true }) as const);

The judge

createJudge(config) returns a function that asks the same questions of every state, one request per state, concurrently (default 8), answers in input order.

  • One request per item. Batching items into one state lets them bleed into each other's judgments. The judge never does that.
  • All-or-nothing. If any request fails, the rest are aborted and the whole invocation is unavailable. One outage never mixes two judges' standards (Jev for half the batch, a fallback for the other half) inside a single decision.
  • Time budget. budgetMs aborts everything in flight and returns over_budget. Use it on request-path gates (a webhook with a 20 s limit).
  • Usage accounting. recordUsage receives one record per invocation with tokens, request count, estimated cost, duration, task name, model, and failed. It is called even when the invocation failed, because tokens spent before a failure were billed. It is not awaited, and a throwing recorder is logged and swallowed: telemetry never fails or delays a gate.
  • No client, no judgment. With TYPESAFE_API_KEY unset the judge returns no_client. It never decides what that means for a gate.

Thresholds

thresholdFromEnv and probabilityThreshold read a number from an environment variable with a validated fallback: garbage, out-of-range, a non-integer where an integer is required, or a probability of exactly 0 all fall back to the default rather than silently disabling the gate. A gate can be loosened or tightened without a deploy.

probabilityAtLeast, probabilityAtMost, expectedLevel, probabilityOf read Score and Choice probability tables. rankIndices and keptIndices turn verdicts into an ordered selection; NaN is never kept.

Reference gates (typesafe-gates/gates)

Each one shipped in production and carries the numbers from the audit that motivated it. Rubrics are generic; put your own subject or community in the context argument.

| Gate | Questions | Policy | Env knob (default) | |---|---|---|---| | relevanceGate | Score, 4 levels: unrelated / same broad category / same niche / directly about | fail-open | RELEVANCE_MIN_PROBABILITY (0.5) on P(level ≥ 2) | | sourceRelevanceGate | Score, 4 levels, judged from a feed's recent titles | fail-open | SOURCE_RELEVANCE_MIN_PROBABILITY (0.5) on P(level ≥ 1) | | candidateGate(categories, fallback) | relevance + newsworthiness (Score) + category (Choice) in one request; rankCandidates picks the top N | fail-open | as above; filler dropped at P ≥ 0.5 | | localityGate | Score, 4 levels: not local / statewide / nearby / hyperlocal; pickLocalFirst builds a local-first pool | fail-open | LOCALITY_MIN_PROBABILITY (0.5), LOCALITY_POOL_TARGET (12, 8 to 30), SOURCE_LOCALITY_MIN_LEVEL (0.5) | | menuPageGate | yes/no: is this page a restaurant's menu | fail-open | MENU_MIN_PROBABILITY (0.4) | | inboundScreenGate | intent (Choice, 7 labels) + AI-directed-instruction injection (yes/no); decideInboundAction is the policy table | fail-closed | fixed: hold at P(injection) ≥ 0.3, auto-reply at P(sales) ≥ 0.7, silent spam at ≥ 0.8; mail longer than the screen sees (subject 300, body 6,000 chars) is held | | factCheckGate | unsupported + contradicts + personal (three yes/no) per section against the evidence the writer was given | fail-open | FACT_CHECK_MIN_PROBABILITY (0.7); contradicts at 0.5; personal note at 0.8 |

Provenance, briefly:

  • Relevance: replaced a batch "return the indices to keep" prompt; the source-level check exists because a bare name once let real but unrelated lyrics and music-chart sources through validation for a wellness creator.
  • Locality: a 14-day audit found 36% of one town's and 23% of another's saved articles were statewide or farther, under a prompt that asked for "hyperlocal". Policy: local first, statewide only as filler, rank rather than hard-drop.
  • Menu page: a heuristic that confirmed pages scoring ≥ 70 was about 59% right over 371 pages. Jev agreed with the extraction outcome about 87% of the time; at 0.4 it would have kept 144 of 147 pages that extracted and dropped 77 of 101 that came back empty.
  • Inbound screen: an auto-responder was acting on any sender's raw email. The screen runs before anything is stored, and an outage holds instead of sending, because the send is the irreversible action.
  • Fact check: a backtest over 148 generated issues found embellished specifics at a low but real rate and invented first-person anecdotes in about 18% of blocks. Precision at 0.7 was roughly half, so it is a reviewer's pointer, not an auto-publish gate, until calibrated against review decisions.

Cost

Jev's published rate at launch is $0.042 per million input tokens, output free (JEV_RATES). A relevance judgment on a title plus a 500-character excerpt is a few hundred tokens, so judging every candidate item is cheaper than one generator call over a batch. Override rates in createJudge when the price changes.

Testing your gates

The judge takes any object with a compatible systemOne method, so tests never touch the network. The package's own tests (test/helpers.ts) show a fake client that honours abort signals, injects failures per call, and adds latency.

Design notes

  • A probability table from a fixed rubric beats free text because the keep rule is a comparison, not a parse.
  • Thresholds live in code and env, never in prompt wording.
  • A gate's policy is part of its definition. Reading "what happens when the judge is down" out of a call site is how an outage silently publishes everything or silently holds everything.
  • Store the expected level, not just the keep decision. Every threshold above was set from a stored score distribution, not from guesswork.

Releasing

  1. Bump version in package.json and add a CHANGELOG.md entry.

  2. Commit, then publish a GitHub release whose tag is vX.Y.Z (a bare X.Y.Z also works) targeting main. The release workflow checks the tag against the version and re-runs lint, typecheck, tests and the build. A release saved as a draft creates no tag and runs nothing.

  3. Publish to npm by hand from a clean checkout of the tagged commit, in a real terminal:

    npm publish --access public

    npm asks for an MFA step with a passkey; approve it in the browser while the command waits. The registry can take a few minutes to show the new version.

The workflow's own publish step (trusted publishing, owner mpointer, repository typesafe-gates, workflow release.yml) fails with 403 OIDC permission denied, because the package requires that passkey step and CI cannot complete it. Releases published by hand carry no provenance attestation.

License

MIT