BayesOutputGate

Universal, zero-runtime-dependency NPM library and CLI that turns the quality scores of an output into a calibrated pass, fail, or escalate decision. Feed the per-dimension scores of any output, from an LLM-as-judge, a classifier, or a regex, and the engine weighs a high-quality hypothesis against a low-quality one with a Bayes Factor on the Jeffreys scale, so you accept, reject, or route to review on principled evidence instead of an arbitrary threshold, for Massive Intelligence (IM) systems and the non-human entities that must validate their own outputs.

BayesOutputGate is the passport control for your outputs. A non-human entity generates an answer, a tool call, or a document, and something has to decide whether it is good enough to ship. Today teams hardcode a threshold, "accept if the judge score is above 0.8", and that number is a guess: it ignores how confidently high-quality and low-quality outputs actually separate, it treats every dimension as equally trustworthy, and it cannot say how sure it is. BayesOutputGate reads the labeled history of known-good and known-bad outputs, calibrates a model of each, and for every new output reports the Bayes Factor between the two hypotheses, then stamps it pass, fail, or escalate with the evidence and the rationale attached.

Core promise: zero required runtime dependencies, a principled Bayes Factor between a high-quality and a low-quality hypothesis rather than a magic threshold, calibrated likelihoods that update online as outcomes arrive, multi-dimension scoring with per-dimension weights, a decision-theoretic policy that minimizes expected loss under asymmetric error costs, goodness-of-fit and dependence diagnostics that tell you when to trust the gate, measured calibration (Brier score, expected calibration error, reliability), a framework-agnostic tool a non-human entity can call to gate its own outputs, a tamper-evident audit trail for compliance, and a node-free core that runs in Node, edge runtimes, and the browser, ESM plus CJS dual distribution.

Why BayesOutputGate

Output validation is a hypothesis-testing question, not a threshold question. An ML engineer says: "Our judge returns a 0 to 1 score. We accept above 0.8. We picked 0.8 because it felt right." A platform engineer says: "We have factuality, safety, and relevance scores. We average them and threshold the average, which lets a great factuality score hide a terrible safety score." A compliance lead says: "When the regulator asks why we shipped this output, all we have is a number and a constant." A fixed threshold answers none of these. The Bayes Factor does: it asks how much more likely the observed scores are under a high-quality hypothesis than under a low-quality one, calibrated from your own labeled data, and it interprets that ratio on a scale that has meant something since Jeffreys.

What sets it apart from a thresholded score and from ad-hoc guardrail rules:

• A Bayes Factor, not a magic number. For each dimension the engine fits a Beta density to the scores of known-high outputs and another to known-low outputs, then the per-dimension density ratio, summed in log space across weighted dimensions, is the combined Bayes Factor. It is interpreted on the Jeffreys scale (the symmetric boundaries 3, 10, 100), so "strong evidence" means strong evidence, not a number you chose.
• Calibrated from history, updated online. Likelihoods are calibrated by method of moments from labeled outputs, regularized by a prior treated as pseudo-observations, and updated online as new labeled outcomes arrive, so the gate gets sharper over time without a full refit.
• Utility-aware decisions. A pure Bayes Factor policy decides on the evidence alone. The decision-theoretic policy converts the Bayes Factor to a posterior probability through an explicit prior, then chooses the action that minimizes expected loss under an asymmetric cost: passing a bad output, rejecting a good one, and escalating to a human each carry their own price. Shipping a wrong medical answer should not cost the same as asking a person to look.
• It tells you when not to trust it, and can act on it. A goodness-of-fit test (Kolmogorov-Smirnov against the fitted Beta) reports whether the score distribution actually matches the model, and a dependence diagnostic measures pairwise correlation between dimensions, because correlated dimensions double-count evidence and inflate the Bayes Factor. Both are measured from your history, and an optional guard makes the gate escalate to human review instead of trusting a Bayes Factor from a misspecified model.
• Calibration is measured, never asserted. Once the gate emits posterior probabilities, the library scores them against realized outcomes with the Brier score, expected calibration error, and a reliability diagram. "Calibrated" is backed by numbers you can compute on your own data.
• A tool a non-human entity can call on itself. The adapter exposes the whole gate as a framework-agnostic tool (name, description, JSON Schema, handler) that drops into an MCP server or an LLM tool-calling API, so a non-human entity (NHE) validates its own outputs before acting on them.
• Proves what it decided. A tamper-evident SHA-256 hash-chained audit trail of every decision, the evidence a compliance review under the EU AI Act asks for.

Install

pnpm add @takk/bayesoutputgate
# or: npm install @takk/bayesoutputgate
# or: yarn add @takk/bayesoutputgate
# or: bun add @takk/bayesoutputgate

The core has zero required runtime dependencies. Every @takk sibling is an optional peer; install only what you compose with.

Quickstart

Calibrate from labeled history, then gate a new output's scores.

import { evaluate, HypothesisManager } from "@takk/bayesoutputgate";

// 1. Calibrate the two hypotheses from labeled outputs (known-high and known-low),
//    each scored across one or more dimensions by any scorer you already have.
const manager = new HypothesisManager();
manager.fit([
  { scores: [{ dimension: "factuality", value: 0.96 }], label: "high" },
  { scores: [{ dimension: "factuality", value: 0.91 }], label: "high" },
  { scores: [{ dimension: "factuality", value: 0.12 }], label: "low" },
  { scores: [{ dimension: "factuality", value: 0.28 }], label: "low" },
]);

// 2. Weigh a new output's scores against the two hypotheses and decide.
const decision = evaluate(
  [{ dimension: "factuality", value: 0.88 }],
  manager.models(),
  { kind: "bayes-factor", passAbove: 10, failBelow: 0.1 },
);

console.log(decision.action);       // "pass" | "fail" | "escalate"
console.log(decision.bayesFactor);  // evidence ratio for high quality over low quality
console.log(decision.strength);     // Jeffreys-scale label, for example "strong-high"
console.log(decision.contributions); // per-dimension breakdown of the evidence

An output passes when the Bayes Factor is at or above passAbove, fails at or below failBelow, and escalates to human review in between. The defaults, 10 and 0.1, are the Jeffreys "strong evidence" boundaries.

Utility-aware decisions

Two errors are not equally costly. Passing a hallucinated medical claim is worse than asking a person to double-check a good answer. The decision-theoretic policy makes the gate minimize expected loss instead of thresholding evidence.

import { evaluate } from "@takk/bayesoutputgate";

const decision = evaluate(scores, models, {
  kind: "decision-theoretic",
  priorHighQuality: 0.8,  // base rate of good outputs, before seeing the scores
  lossFalsePass: 10,      // cost of shipping a low-quality output
  lossFalseFail: 1,       // cost of rejecting a high-quality output
  escalationCost: 0.5,    // cost of routing to a human reviewer
});

console.log(decision.action);               // the expected-loss-minimizing action
console.log(decision.posteriorHighQuality); // P(high quality | the scores)
console.log(decision.expectedLoss);         // { pass, fail, escalate }
console.log(decision.rationale);            // "expected-loss"

The Bayes Factor becomes a posterior through your prior (posterior log-odds equal the log-Bayes-Factor plus the prior log-odds), and the action with the lowest expected loss wins. Raise lossFalsePass and the gate becomes conservative; raise escalationCost and it sends fewer outputs to review.

Multi-dimension scoring and the dependence diagnostic

Real outputs are scored on several axes, factuality, fluency, safety, relevance, and they should not count equally. Each dimension carries a weight in the combined Bayes Factor, and the engine warns when two dimensions are correlated enough to double-count evidence.

import { dependenceDiagnostic, HypothesisManager } from "@takk/bayesoutputgate";

const manager = new HypothesisManager({
  dimensions: [
    { dimension: "factuality", weight: 2 },  // weigh factuality twice as heavily
    { dimension: "safety", weight: 3 },       // and safety more still
    { dimension: "fluency", weight: 1 },
  ],
});
manager.fit(labeledHistory);

// Are any dimensions correlated enough that the independence assumption is unsafe?
const dependence = dependenceDiagnostic(labeledHistory.map((o) => o.scores));
console.log(dependence.independenceAssumptionSafe); // false if a pair crosses the threshold
console.log(dependence.flagged);                    // the offending pairs, by Pearson correlation

The combined Bayes Factor sums per-dimension log-density differences, which assumes the dimensions are conditionally independent given the hypothesis. When independenceAssumptionSafe is false, drop or merge the correlated dimensions, or down-weight them, so the gate does not over-count.

Calibration, measured not asserted

The gate is only as good as the model behind it. Two measurements tell you whether to trust it: goodness-of-fit asks whether the Beta assumption holds for your scores, and decision calibration asks whether the posterior probabilities match reality.

import {
  brierScore,
  expectedCalibrationError,
  goodnessOfFit,
  reliability,
} from "@takk/bayesoutputgate";

// 1. Does the per-dimension Beta assumption hold? (Kolmogorov-Smirnov against the fit.)
const fit = goodnessOfFit(highQualityScores, { a: 9, b: 2 });
console.log(fit.adequate); // false means the density-ratio likelihood should not be trusted here

// 2. Do the posterior probabilities match observed outcomes?
const predictions = [
  { probability: 0.92, outcome: 1 },
  { probability: 0.18, outcome: 0 },
  // ...one row per decided output, outcome 1 if it was truly high quality, else 0
];
console.log(brierScore(predictions));              // mean squared error, lower is better
console.log(expectedCalibrationError(predictions)); // gap between predicted and empirical rates
console.log(reliability(predictions));              // the reliability diagram, bin by bin

When goodness-of-fit fails, the Bayes Factor is optimizing the wrong model and no decision metric will reveal it, which is exactly why this check comes first.

A gate that recalibrates and seals every decision

OutputGateMonitor wraps the gate with online recalibration and a tamper-evident audit chain, so every decision a non-human entity acts on is explainable after the fact. With guards, it assesses its modeling assumptions on every fit and escalates rather than trusting a Bayes Factor from a misspecified model.

import { OutputGateMonitor } from "@takk/bayesoutputgate";

const monitor = new OutputGateMonitor({
  policy: { kind: "bayes-factor", passAbove: 10, failBelow: 0.1 },
  guards: { requireGoodnessOfFit: true, requireIndependence: true },
});
monitor.fit(labeledHistory);

console.log(monitor.assumptionReport); // goodness-of-fit and dependence, assessed from the history

const { decision, entry } = await monitor.record(
  [{ dimension: "factuality", value: 0.88 }],
  { timestamp: "2026-06-24T00:00:00Z" },
);

console.log(decision.action);    // "escalate" with rationale "assumption-violated" if a guard tripped
console.log(decision.assumptions); // the report attached to the decision
console.log(entry.hash);         // the seal over this decision

// Later, when a true label arrives, fold it back in to sharpen calibration.
monitor.observe({ scores: [{ dimension: "factuality", value: 0.88 }], label: "high" });

You can also guard a stateless evaluate by passing a precomputed report from assessAssumptions(history, models): when requireGoodnessOfFit or requireIndependence is set and the matching assumption fails, the decision is forced to escalate with rationale "assumption-violated", and the evidence is preserved on the decision for the reviewer.

A tool a non-human entity can call

The adapter turns the whole gate into a framework-agnostic tool. The shape matches what MCP servers and LLM tool-calling APIs expect, so a non-human entity (NHE) can validate its own output before acting on it. Input arriving from a model is parsed defensively, and the output is made JSON-safe.

import { bayesOutputGateTool, runTool } from "@takk/bayesoutputgate";

bayesOutputGateTool.name;        // "bayes_output_gate"
bayesOutputGateTool.inputSchema; // JSON Schema for the tool input

const output = runTool({
  scores: [{ dimension: "factuality", value: 0.88 }],
  models: [{ dimension: "factuality", high: { a: 9, b: 2 }, low: { a: 2, b: 9 }, weight: 1 }],
  policy: { kind: "bayes-factor", passAbove: 10, failBelow: 0.1 },
});
console.log(output.action); // "pass" | "fail" | "escalate"

Entry points

Thirteen subpath exports, each importable on its own. The core is node-free; only node touches a Node built-in.

| Import | What it gives you | |---|---| | @takk/bayesoutputgate | The evaluate gate, OutputGate, OutputGateMonitor, plus the full toolkit and types. | | @takk/bayesoutputgate/beta | The calibrated BetaModel, the online-updatable engine both hypotheses share. | | @takk/bayesoutputgate/hypothesis | The HypothesisManager, per-dimension high and low score models from labeled history. | | @takk/bayesoutputgate/likelihood | Per-score and marginal likelihoods, density-ratio and Beta-Binomial modes. | | @takk/bayesoutputgate/bayesfactor | bayesFactor and the Jeffreys-scale interpretation. | | @takk/bayesoutputgate/dimensions | dependenceDiagnostic, pairwise correlation between dimensions. | | @takk/bayesoutputgate/policy | decide and posteriorHighQuality, the Bayes Factor and decision-theoretic policies. | | @takk/bayesoutputgate/calibration | goodnessOfFit, brierScore, expectedCalibrationError, reliability, assessAssumptions. | | @takk/bayesoutputgate/gate | evaluate, OutputGate, and OutputGateMonitor, with optional assumption guards. | | @takk/bayesoutputgate/audit | Append-only SHA-256 hash-chained audit log, via Web Crypto. | | @takk/bayesoutputgate/adapter | The framework-agnostic MCP and LLM tool definition. | | @takk/bayesoutputgate/node | File loaders for JSON and CSV score history, over node:fs. | | @takk/bayesoutputgate/edge | The node-free core, re-exported for edge runtimes and the browser. |

Tamper-evident audit trail

import { AuditChain, verifyChain } from "@takk/bayesoutputgate";

const chain = new AuditChain();
await chain.append({ action: "pass", bayesFactor: 42.1 });
await chain.append({ action: "escalate", bayesFactor: 4.3 });

await verifyChain(chain.toArray()); // { valid: true }, until any entry is altered

Each entry is recorded append-only and chained to the previous one through a SHA-256 hash of its canonical form. Any later edit, deletion, or reordering breaks the chain and verifyChain reports the first broken index. The seal is an integrity seal, not a digital signature: it makes tampering detectable, not impossible. The chain uses the Web Crypto API, so the audit surface runs in Node, edge runtimes, and the browser. The OutputGateMonitor records every decision into such a chain automatically.

Governance

Every decision can be recorded and replayed. The OutputGateMonitor appends each verdict to a tamper-evident audit chain, so every output a non-human entity ships or rejects is explainable after the fact, with the Bayes Factor and the policy it was decided on. No runtime dependency is taken; you wire the chain to your own telemetry and compliance stack. This is the governance seam for Massive Intelligence (IM) systems and the validation evidence the EU AI Act expects.

CLI

BayesOutputGate ships a command-line tool that runs the real engine, with every number produced by execution.

# Fit the two hypotheses from labeled history, then decide each output's scores.
npx @takk/bayesoutputgate gate history.json scores.json

# Compute the Bayes Factor for each output against explicit dimension models.
npx @takk/bayesoutputgate bayes-factor models.json scores.json

# Fit, then report goodness-of-fit per dimension and the dependence diagnostic.
npx @takk/bayesoutputgate calibrate history.json

# Verify the integrity of a sealed decision audit chain.
npx @takk/bayesoutputgate audit-verify chain.json

History is a JSON array of labeled observations ({ scores, label }); scores is a JSON array of score vectors, or a CSV with a header row of dimension names and one row of values per output. Flags: --pass-above, --fail-below, --json. Exit codes: 0 ok, 2 usage or input error, 30 a fail decision, 40 an escalate decision, 20 a broken audit chain.

The math, in one paragraph

For each quality dimension the engine models the score of a high-quality output and the score of a low-quality output as two Beta densities, calibrated from labeled history by method of moments and regularized by a prior treated as pseudo-observations. The Bayes Factor for an output is the ratio of the likelihood of its scores under the high-quality hypothesis to the likelihood under the low-quality one; across dimensions the log-Bayes-Factors add, weighted, which is the conditional-independence assumption the dependence diagnostic checks. The combined log-Bayes-Factor lands on the Jeffreys scale (Kass and Raftery, 1995), whose symmetric boundaries at 3, 10, and 100 separate inconclusive, substantial, strong, and decisive evidence. A pure policy thresholds the Bayes Factor on that scale; the decision-theoretic policy turns it into a posterior through an explicit prior (posterior log-odds equal the log-Bayes-Factor plus the prior log-odds) and picks the expected-loss-minimizing action under an asymmetric loss. Calibration is measured, not assumed: goodness-of-fit by the Kolmogorov-Smirnov statistic against the fitted Beta, and decision calibration by the Brier score, expected calibration error, and a reliability diagram. The data is yours; the calibration is the library's.

Benchmark

A value benchmark scores the gate against fairly-tuned fixed thresholds on a held-out test set the gate never calibrates on, under an asymmetric loss (a false pass costs 10, a false fail 2, an escalation 1). Each output is two-dimensional, and a "masking" low-quality output scores high on factuality but low on safety, the failure a single averaged threshold hides. Every number is from real execution against the compiled dist; run it with node --import tsx benchmarks/value.ts.

| Regime | Policy | Avg loss | Accuracy | Escalated | |---|---|---|---|---| | Beta-satisfied | fixed threshold (best of mean, min) | 0.619 | 78.6% | 0% | | Beta-satisfied | BayesOutputGate | 0.456 | 93.7% | 23.7% | | Beta-violated | fixed threshold (best of mean, min) | 0.986 | 50.8% | 0% | | Beta-violated | BayesOutputGate (unguarded) | 0.869 | 79.7% | 46.2% |

When the Beta assumption holds, the calibrated multi-dimension gate cuts average loss about 26 percent below the best tuned threshold by escalating the genuinely ambiguous cases to review. When the assumption is violated (a strongly bimodal score distribution), the goodness-of-fit diagnostic detects it, a signal a fixed threshold has no way to produce; the gate degrades gracefully, keeping the lowest loss and zero false passes, and with the goodness-of-fit guard it escalates every output rather than acting on a model it knows is misspecified. The gate sustains roughly 2.7 million two-dimension decisions per second. The point is not that the gate always wins, it is that the gate knows when to defer and tells you when its model does not fit.

See examples/ for five runnable demos against the compiled dist: the basic gate, the decision-theoretic policy, the assumption guards, the MCP tool adapter, and node-free edge usage with a verifiable audit chain.

Quality

• 90 tests across 15 suites, all passing under Vitest, including the special functions verified against closed forms, the Beta fit and online update, the Bayes Factor against the Jeffreys boundaries, the decision-theoretic policy verified against expected loss, the goodness-of-fit and dependence diagnostics, the assumption guards that escalate on a violated assumption, and the audit chain detecting tampering and broken links.
• Coverage: statements 94.64%, branches 87.58%, functions 96.24%, lines 95.91%.
• Lint clean under Biome.
• Typecheck clean under TypeScript in maximum strict mode (exactOptionalPropertyTypes, useUnknownInCatchVariables, noUncheckedIndexedAccess, noPropertyAccessFromIndexSignature).
• publint clean and @arethetypeswrong/cli green across all thirteen subpaths.
• size-limit under budget on every bundle (brotli core facade 5.29 kB).
• Distribution smoke test exercising the compiled ESM and CJS artifacts and the compiled CLI spawned as a single Node process. The full suite, the smoke test, and the CLI are re-verified on Node 24.
• Zero required runtime dependencies; a node-free core that runs in Node, edge runtimes, and the browser; dual ESM plus CJS; TypeScript-first.

See SPEC.md for the formal specification, public surface, and stability promise.

FAQ

How is this different from thresholding a judge score? A threshold compares one number to a constant you guessed. BayesOutputGate compares how likely the observed scores are under a high-quality hypothesis against a low-quality one, calibrated from your labeled history, and reports the strength of that evidence on the Jeffreys scale. It accounts for how well good and bad outputs actually separate, weighs dimensions, and can express its uncertainty by escalating instead of forcing a binary call.

What if my dimensions are correlated? The dependence diagnostic measures pairwise correlation across your history and flags pairs that cross a threshold. Correlated dimensions double-count evidence and inflate the Bayes Factor, so when a pair is flagged you drop, merge, or down-weight one of them. The check is honest about a real limitation rather than hiding it.

Does it act on the decision? No. BayesOutputGate returns a recommendation, pass, fail, or escalate, with the Bayes Factor, the strength, and the per-dimension contributions. Your orchestrator (an MCP server, an LLM tool loop, a Hermes-style runtime, your own controller) consumes that decision and acts. The gate decides; you wire the action.

Is the calibration real? It is measured. The library computes goodness-of-fit against the fitted Beta and, once outcomes are known, the Brier score, expected calibration error, and a reliability diagram on your own data. "Calibrated" means you can verify it, not that the library asserts it.

Bayes Factor policy or decision-theoretic? Use the Bayes Factor policy when the two error types cost about the same and you just want principled evidence thresholds. Use the decision-theoretic policy when passing a bad output and rejecting a good one have different costs, which is the usual case in production.

Does this work in Cloudflare Workers, Vercel Edge, Bun, and Deno? Yes. The core is node-free; the audit seal uses the Web Crypto API, not node:crypto. Import @takk/bayesoutputgate or @takk/bayesoutputgate/edge anywhere with Web Crypto. Only @takk/bayesoutputgate/node requires Node.

Contributing

See .github/CONTRIBUTING.md for the contributor guide. Substantive proposals open a GitHub Issue first; trivial fixes can go straight to a PR. All commits require DCO sign-off (git commit -s). Non-trivial contributions are governed by the Contributor License Agreement.

Community and support

• Issues and feature requests. Open a GitHub issue at davccavalcante/bayesoutputgate/issues. Include the package version, a minimal reproduction, the scores and models you fed in, and where relevant the evaluate() decision.
• Security disclosures. Do not open public issues for vulnerabilities. Follow the responsible-disclosure flow in SECURITY.md, contact [email protected] (or [email protected]) with the [SECURITY] prefix.
• Code of Conduct. This project follows the Contributor Covenant 2.1. Participation in any BayesOutputGate space implies agreement.
• Contributions. All non-trivial contributions go through the Contributor License Agreement. Tests, lint, typecheck, and build must be green before review (pnpm verify).

Author

Created by David C Cavalcante, [email protected] (preferred), [email protected] (Takk relay), linkedin.com/in/hellodav, x.com/davccavalcante, takk.ag.

BayesOutputGate is part of a broader portfolio of NPM packages targeting Massive Intelligence (IM) native infrastructure for 2026-2030, built at Takk Innovate Studio. Calibrated output validation with a Bayes Factor is a strong default for that era, when fleets of long-running non-human entities must validate their own outputs before acting on them.

Related research by the author

The architectural philosophy behind BayesOutputGate, separating the calibrated models, the Bayes Factor, the decision policy, the calibration measurement, and persistence into composable, independently-governed layers, echoes the author's research frameworks:

• MAIC (Massive Artificial Intelligence Consciousness), a systemic intelligence framework designed to coordinate, supervise, and govern large-scale Massive Intelligence ecosystems, providing global context awareness, alignment, and orchestration across multiple models, agents, and decision layers.
• HIM (Hybrid Entity Intelligence Model), a hybrid intelligence layer that integrates Massive Intelligence systems with human-defined logic, rules, heuristics, and strategic intent, interpreting objectives and structuring decision-making before and after model execution.
• NHE (Noumenal Higher-order Entity), a non-human cognitive entity with a defined functional identity and operational agency within a Massive Intelligence ecosystem, operating through coordinated intelligence layers while maintaining a non-anthropomorphic identity.

These frameworks are published independently of BayesOutputGate and are separate works:

• Research papers: The Soul of the Machine, Beyond Consciousness in LLMs, The Cave of Silence.
• PhilPapers profile: David Cortes Cavalcante.
• Hugging Face: TeleologyHI.
• GitHub: davccavalcante, Takk8IS.

Sponsors

Join the journey as the portfolio continues to ship Massive Intelligence (IM) native infrastructure. Your support is the cornerstone of this work.

• Sponsor on GitHub: github.com/sponsors/davccavalcante
• USDT (TRC-20): TS1vuhMAhFpbd7y68cu5ZtP9PsXVmZWmeh

Privacy

BayesOutputGate runs entirely inside your own process and infrastructure. It makes no outbound calls to the author, collects no telemetry, and ships no analytics. The only state it holds is the labeled history, scores, and decisions you feed it. See PRIVACY.md for the full data-handling notice, including how the optional file loaders read history from disk.

License

Licensed under the Apache License 2.0. See LICENSE for the full text and NOTICE for attribution and third-party component licenses. You may use, modify, and distribute the code under the terms of that license, including its patent grant and attribution requirements.