BayesDecide

Universal, zero-runtime-dependency NPM library and CLI for Bayesian multi-armed bandits over prompt variants. Thompson sampling routes more traffic to the variant that looks best while staying calibrated, and a stopping rule promotes a winner without waiting for a fixed-N A/B test, for Massive Intelligence (IM) systems.

BayesDecide is the decision layer for continuous prompt experimentation. It holds a posterior over the reward of each prompt variant and treats every query as a chance to both learn and exploit. Thompson sampling draws one sample from each variant's posterior and serves the variant with the best draw, so traffic shifts toward the leading variant in proportion to how confident the evidence is, not all at once and not by a fixed split. Each outcome folds back in with a closed-form conjugate update. A sequential stopping rule watches the posterior probability that a variant is best and tells you when the evidence is decisive enough to promote a winner, rather than waiting for a pre-committed sample size.

Core promise: zero required runtime dependencies, single-function setup, exact closed-form inference with O(1) updates and zero inference latency, a node-free core that runs in Node, edge runtimes, and the browser, ESM plus CJS dual distribution, and a framework-agnostic adapter that binds to any generation backend with no hard dependency.

Why BayesDecide

Choosing the better prompt is a decision under uncertainty: serve more of the variant that is winning, but keep enough exploration to be sure you are right. A fixed-N A/B test answers this badly. It splits traffic evenly for a sample size chosen in advance, pays full regret on the losing variant for the entire run, and forbids peeking at the result before the end on pain of inflating the false-positive rate. BayesDecide replaces the even split and the fixed N with the correct primitive: an explicit prior, continuous updates from evidence, Thompson sampling that explores in proportion to uncertainty, and a sequential stopping rule you can read at any time.

What sets it apart from a uniform A/B test:

• Routes by posterior, not by a fixed split. Thompson sampling sends a query to a variant with probability equal to that variant's posterior probability of being best, so a clearly leading variant gets most of the traffic while a close race keeps exploring. Regret being logarithmic is a theoretical property of Thompson sampling, not a guarantee this package enforces.
• Promotes on evidence, not on a pre-committed N. The default stopping rule reports a winner once the posterior probability that it is best clears your confidence threshold. This is a Bayesian decision heuristic, a sound and practical default, not an error-controlled test under repeated peeking.
• Offers an error-controlled mode when you need it. Set stopping: "anytime-valid" for a confidence-sequence test whose error rate holds under unlimited peeking. It is the rigorous counterpart to the default heuristic, available for the Bernoulli reward model.
• Exact, not approximate. The credible intervals and best-arm probabilities are computed from the true Beta posterior, the regularized incomplete beta via continued fraction and its inverse via bisection, not a normal approximation.
• Keeps a human or policy in the loop. The promotion engine is advisory by design: it returns the winning variant when the experiment is decisive, and never auto-routes traffic or mutates the experiment on its own. Acting on the winner is your call.
• Handles bounded and unbounded rewards. A Beta-Bernoulli arm for rewards in [0, 1], binary outcomes or fractional Massive Intelligence (IM)-as-judge scores, and a Gaussian arm for unbounded metrics like latency or cost.
• Proves what happened. A tamper-evident SHA-256 hash-chained audit trail of decisions, the evidence a review asks for when a promotion is questioned.

Install

pnpm add @takk/bayesdecide
# or: npm install @takk/bayesdecide
# or: yarn add @takk/bayesdecide
# or: bun add @takk/bayesdecide

The core has zero required runtime dependencies. Every @takk sibling is an optional peer; install only what you compose with.

Quickstart

import { createExperiment } from "@takk/bayesdecide";

const experiment = createExperiment({
  variants: [{ id: "concise" }, { id: "detailed" }],
  confidence: 0.95,
});

const { variant } = experiment.select();   // Thompson sampling picks a variant
// ... serve the prompt for `variant`, get an outcome ...
experiment.observe(variant, reward);        // reward in [0, 1]

const decision = experiment.decide();       // { done, best, probabilityBest, reason }
if (decision.done) console.log("promote", decision.best);

Every variant starts at the uniform prior Beta(1, 1). select shifts toward the leading variant as evidence accrues, and decide reports a winner as soon as the posterior probability of being best clears your confidence.

Quickstart, the one-call loop

When you can run and score a variant inline, pull is the whole loop in a single call: it selects a variant, runs it, scores the output, and folds the reward back in.

import { createExperiment } from "@takk/bayesdecide";

const experiment = createExperiment({
  variants: [{ id: "concise" }, { id: "detailed" }],
});

const { variant, output, reward } = await experiment.pull(
  (v) => generateAnswer(promptFor(v)),     // run the chosen variant
  (answer) => scoreAnswer(answer),         // return a reward in [0, 1]
);

The score callback is where a judge, a rubric, a thumbs-up signal, or a measured metric becomes a reward. Anything in [0, 1] works for the default Bernoulli arm.

Prompt-router adapter

The adapter is the integration seam. It is dependency injected on purpose: it takes a generate function instead of importing any model SDK, so it binds an experiment to the Vercel AI SDK, Mastra, Genkit, a raw fetch, or a non-human entity's own generation loop without a single hard dependency. A router maps variant ids to prompt payloads, asks the experiment which to serve, runs it, scores the output, and observes the reward.

import { createExperiment } from "@takk/bayesdecide";
import { createPromptRouter } from "@takk/bayesdecide/adapter";

const experiment = createExperiment();
const router = createPromptRouter({
  experiment,
  prompts: {
    concise: "Answer in one sentence.",
    detailed: "Answer thoroughly, with examples.",
  },
});

const { variant, output, reward } = await router.run(
  (prompt) => callModel(prompt),           // any backend, injected
  (answer) => judge(answer),               // reward in [0, 1]
);

This is the loop a non-human entity (NHE) closes over the prompt variants it generates: a complement to systems that mutate prompts, this one decides among them under uncertainty.

Unbounded metrics, the Gaussian arm

The Beta-Bernoulli arm models a reward in [0, 1]. For a metric without a natural ceiling, latency, cost, or a score whose variance matters, use the Normal arm. Set rewardModel: "normal" and observe the raw continuous metric.

import { createExperiment } from "@takk/bayesdecide";

const experiment = createExperiment({
  rewardModel: "normal",
  variants: [{ id: "model-a" }, { id: "model-b" }],
});

experiment.observe("model-a", 142);   // raw metric, not normalized to [0, 1]
experiment.observe("model-b", 318);

For unbounded metrics, reach for the Normal arm; the Beta arm is for rewards confined to [0, 1].

Entry points

Sixteen subpath exports, each importable on its own. The core is node-free; only node touches a Node built-in.

| Import | What it gives you | |---|---| | @takk/bayesdecide | The createExperiment facade wiring everything below, plus the full toolkit. | | @takk/bayesdecide/beta | The Beta distribution: priors, moments, cdf, quantile, credible interval, conjugate update, sampling. | | @takk/bayesdecide/normal | The Normal arm: a Normal-inverse-Gamma posterior for unbounded continuous rewards. | | @takk/bayesdecide/sampler | The selection strategies: Thompson sampling and greedy. | | @takk/bayesdecide/bestarm | Best-arm identification: Monte Carlo probability that each variant is best. | | @takk/bayesdecide/sequential | The stopping rules: the Bayesian heuristic and the anytime-valid confidence sequence. | | @takk/bayesdecide/promotion | The advisory promotion engine: returns a winner, never routes traffic. | | @takk/bayesdecide/store | The posterior store keyed by (variant, cohort), and portable JSON snapshots. | | @takk/bayesdecide/registry | The variant registry: register, label, and enumerate variants. | | @takk/bayesdecide/decay | Exponential time-decay of stale evidence back toward the prior. | | @takk/bayesdecide/evaluator | Reward evaluation helpers for turning outcomes into rewards. | | @takk/bayesdecide/calculator | The acceleration calculator: how much faster the bandit reaches a decision than a uniform A/B test. | | @takk/bayesdecide/adapter | createPromptRouter, the framework-agnostic generation seam. | | @takk/bayesdecide/audit | Append-only SHA-256 hash-chained audit log of decisions, via Web Crypto. | | @takk/bayesdecide/node | Durable file-backed persistence with atomic writes. | | @takk/bayesdecide/edge | The node-free core, re-exported for edge runtimes and the browser. |

Two stopping rules

The default rule is a Bayesian decision heuristic: stop and promote once the posterior probability that the leading variant is best clears your confidence. It is sound and practical, the right default for most experiments, but its error rate is not controlled under repeated peeking, you simply read it whenever you like and act on the probability it reports.

const experiment = createExperiment({ confidence: 0.95 });
// ... observe outcomes ...
const decision = experiment.decide();
// { done, best, probabilityBest, reason }

When you need a guarantee that holds no matter how often you check, switch to the anytime-valid mode. It runs a confidence-sequence test whose error rate is valid under unlimited peeking, the error-controlled counterpart to the default heuristic. It supports the Bernoulli reward model and requires time-decay disabled.

const experiment = createExperiment({
  stopping: "anytime-valid",
  confidence: 0.95,   // alpha defaults to 1 - confidence
});

Per-cohort context

Pass a cohort to select, observe, and decide to run an independent experiment per cohort, the discrete form of a contextual bandit. Each cohort keeps its own posteriors, so a variant that wins for one segment can lose for another without the two averaging into one misleading number.

experiment.select("enterprise");
experiment.observe("concise", 1, "enterprise");
experiment.decide("enterprise");

This is categorical, per-cohort context. Feature-based contextual bandits, where the context is a continuous feature vector, are on the roadmap.

Time-decay

Evidence goes stale. With decay enabled, older observations relax exponentially back toward the prior, so the experiment tracks a variant whose true reward drifts, after a model upgrade or a prompt edit, instead of being anchored to history. Disabled by default; note that the anytime-valid stopping mode requires it off.

const experiment = createExperiment({
  decay: { halfLifeMs: 7 * 24 * 60 * 60 * 1000 },   // one week
});

Tamper-evident audit trail

import { AuditLog } from "@takk/bayesdecide/audit";

const log = new AuditLog();
await log.append({
  variant: "concise", reward: 1,
  probabilityBest: 0.97, decision: "promote", at: Date.now(),
});

await log.verify(); // true, until any entry is altered

Each decision is recorded append-only and chained to the previous one through a SHA-256 hash of its canonical form. Any later edit, deletion, or reordering breaks the chain and verify returns false. The seal is an integrity seal, not a digital signature: it proves the record was not tampered with, and makes tampering detectable, not that the record is unalterable, and not who produced it. The chain uses the Web Crypto API, so the audit surface runs in Node, edge runtimes, and the browser.

Observability and governance

Every decision is observable. Pass an observer and the experiment notifies you of each selection, each observation, and each stopping decision, so you can emit an OpenTelemetry span per query, ship the decision to a compliance log, or aggregate across a fleet. The experiment never lets an observer error break a decision, so it is safe to wire production telemetry through it.

import { createExperiment } from "@takk/bayesdecide";

const experiment = createExperiment({
  observer: {
    onObserve: (variant, cohort, reward) => metrics.record(variant, cohort, reward),
    onDecision: (decision, cohort) => {
      span.addEvent("bandit.decision", {
        cohort,
        done: decision.done,
        best: decision.best ?? "none",
        probabilityBest: decision.probabilityBest,
        reason: decision.reason,
      });
    },
  },
});

Together with the tamper-evident audit trail, the observer is the governance seam for Massive Intelligence (IM) systems: every promotion a non-human entity makes is explainable after the fact, with the probability and the reason it was decided on. No runtime dependency is taken; you wire it to your own telemetry stack.

CLI

BayesDecide ships a command-line tool that runs the real experiment, with every number produced by execution.

# Simulate a bandit over four variants for 2000 queries and report how it
# routed traffic, the winner it promoted, and the regret it paid against a
# uniform A/B test.
npx @takk/bayesdecide simulate --n 2000 --variants 4

# Replay a sequence of rewards (1 success, 0 failure) and print the learned
# posteriors and the stopping decision.
echo "1 1 0 1 0 0 0" | npx @takk/bayesdecide replay

See examples/ for runnable demos, including an agent-loop, non-human-entity demo, and benchmarks/ for the paired-stream regret benchmark.

The math, in one paragraph

Each variant in a cohort has a prior over its reward: Beta(alpha, beta) for the Bernoulli model, Normal-inverse-Gamma for the Normal model. An observed reward updates the posterior in closed form, O(1) per update with zero inference latency. To select, Thompson sampling draws one sample from each variant's posterior and serves the variant with the best draw, so a variant is chosen with probability equal to its posterior probability of being best; greedy serves the posterior mean instead. The probability that each variant is best is estimated by Monte Carlo over the posteriors. The default stopping rule promotes when that probability clears confidence, a Bayesian decision heuristic; the anytime-valid rule instead runs a confidence-sequence test whose error rate holds under unlimited peeking. Priors are explicit and customizable per variant; the reward is always yours, the calibration is always the library's.

Benchmark

benchmarks/ runs the bandit and a uniform A/B test over the same reward streams, with both facing identical luck so the only variable is the routing rule, and reports cumulative regret, the reward lost by not always serving the best variant. Across scenarios the bandit cuts a uniform A/B test's regret by 66 to 99.5 percent. For the two-variant, large-gap case:

| Policy | Regret | |---|---| | uniform A/B test | 800 | | BayesDecide (Thompson) | 4.4 |

The bandit pays almost no regret in a clear race because it routes away from the losing variant within the first few dozen queries, while the uniform test keeps paying the full gap on half of all traffic until the pre-committed N is reached. Run it yourself: pnpm run build && node benchmarks/.

Quality

• 106 tests across 22 suites, all passing under Vitest.
• Coverage: statements 96.21%, branches 91.36%, functions 96.89%, lines 97.56%.
• Lint clean under Biome.
• Typecheck clean under TypeScript in maximum strict mode (exactOptionalPropertyTypes, useUnknownInCatchVariables, noUncheckedIndexedAccess).
• publint clean and @arethetypeswrong/cli green across all sixteen subpaths.
• size-limit under budget on every bundle (brotli core 4.63 kB).
• Distribution smoke test exercising the compiled ESM and CJS artifacts and the compiled CLI spawned as a single Node process.
• Zero required runtime dependencies; a node-free core that runs in Node, edge runtimes, and the browser; dual ESM plus CJS; TypeScript-first.

See SPEC.md for the formal specification, public surface, and stability promise.

FAQ

Why not just run a fixed-N A/B test? A uniform A/B test splits traffic evenly for a sample size chosen in advance, so it pays full regret on the losing variant for the whole run and forbids reading the result early without inflating the false-positive rate. BayesDecide routes by the posterior, so traffic moves toward the winner as evidence accrues, and the default stopping rule lets you read a decision whenever you like. When you need an error guarantee under repeated peeking, the anytime-valid mode provides it.

Is the default stopping rule a hypothesis test? No. The default is a Bayesian decision heuristic: it reports the posterior probability that the leading variant is best and promotes once that clears your confidence. It is sound and practical, but its error rate is not controlled under repeated peeking. For that guarantee, use stopping: "anytime-valid", the error-controlled confidence-sequence counterpart, available for the Bernoulli model.

Does the promotion engine switch traffic for me? No, and that is deliberate. The promotion engine is advisory: it returns the winning variant when the experiment is decisive and otherwise null. It never auto-routes traffic or mutates the experiment, so a human or a policy stays in the loop on the actual switch.

Bounded or unbounded rewards? The Beta-Bernoulli arm handles a reward in [0, 1], binary outcomes or fractional judge scores. For an unbounded metric like latency or cost, set rewardModel: "normal" and observe the raw value with the Gaussian arm.

Does this work in Cloudflare Workers, Vercel Edge, Bun, and Deno? Yes. The core is node-free; the audit seal uses the Web Crypto API, not node:crypto. Import @takk/bayesdecide or @takk/bayesdecide/edge anywhere with Web Crypto. Only @takk/bayesdecide/node requires Node.

Where does the state live? By default, in-process memory, with portable JSON snapshots via snapshot() and load(). For durability, use the file-backed store from @takk/bayesdecide/node. For an edge runtime, snapshot to your own KV store between invocations.

Contributing

See .github/CONTRIBUTING.md for the contributor guide. Substantive proposals open a GitHub Issue first; trivial fixes can go straight to a PR. All commits require DCO sign-off (git commit -s). Non-trivial contributions are governed by the Contributor License Agreement.

Community and support

• Issues and feature requests. Open a GitHub issue at davccavalcante/bayesdecide/issues. Include the package version, a minimal reproduction, expected versus actual behavior, and where relevant the decide() output.
• Security disclosures. Do not open public issues for vulnerabilities. Follow the responsible-disclosure flow in SECURITY.md, contact [email protected] (or [email protected]) with the [SECURITY] prefix.
• Code of Conduct. This project follows the Contributor Covenant 2.1. Participation in any BayesDecide space implies agreement.
• Contributions. All non-trivial contributions go through the Contributor License Agreement. Tests, lint, typecheck, and build must be green before review (pnpm verify).

Author

Created by David C Cavalcante, [email protected] (preferred), [email protected] (Takk relay), linkedin.com/in/hellodav, x.com/davccavalcante, takk.ag.

BayesDecide is part of a broader portfolio of NPM packages targeting Massive Intelligence (IM) native infrastructure for 2026-2030, built at Takk Innovate Studio. Bandit-driven prompt decisioning is a strong default for that era.

Related research by the author

The architectural philosophy behind BayesDecide, separating inference, decision, and persistence into composable, independently-governed layers, echoes the author's research frameworks:

• MAIC (Massive Artificial Intelligence Consciousness), a systemic intelligence framework designed to coordinate, supervise, and govern large-scale Massive Intelligence ecosystems, providing global context awareness, alignment, and orchestration across multiple models, agents, and decision layers.
• HIM (Hybrid Entity Intelligence Model), a hybrid intelligence layer that integrates Massive Intelligence systems with human-defined logic, rules, heuristics, and strategic intent, interpreting objectives and structuring decision-making before and after model execution.
• NHE (Noumenal Higher-order Entity), a non-human cognitive entity with a defined functional identity and operational agency within a Massive Intelligence ecosystem, operating through coordinated intelligence layers while maintaining a non-anthropomorphic identity.

These frameworks are published independently of BayesDecide and are separate works:

• Research papers: The Soul of the Machine, Beyond Consciousness in LLMs, The Cave of Silence.
• PhilPapers profile: David Cortes Cavalcante.
• Hugging Face: TeleologyHI.
• GitHub: davccavalcante, Takk8IS.

Sponsors

Join the journey as the portfolio continues to ship Massive Intelligence (IM) native infrastructure. Your support is the cornerstone of this work.

• Sponsor on GitHub: github.com/sponsors/davccavalcante
• USDT (TRC-20): TS1vuhMAhFpbd7y68cu5ZtP9PsXVmZWmeh

Privacy

BayesDecide runs entirely inside your own process and infrastructure. It makes no outbound calls to the author, collects no telemetry, and ships no analytics. The only state it holds is the reward evidence you feed it. See PRIVACY.md for the full data-handling notice, including how the optional file store persists state on disk.

License

Licensed under the Apache License 2.0. See LICENSE for the full text and NOTICE for attribution and third-party component licenses. You may use, modify, and distribute the code under the terms of that license, including its patent grant and attribution requirements.