@framers/agentos-ext-grounding-guard

NLI-based factual grounding verification for @framers/agentos. Extracts claims from agent outputs and verifies each against the supplied source documents using natural-language-inference scoring.

What it does

Extracts each factual claim from an agent response, runs it against retrieval context (or supplied passages) using an NLI model, and reports per-claim entailment scores. Claims below the configured entailment threshold are flagged as ungrounded.

Use it to:

• Reduce hallucination in RAG agents
• Enforce a minimum-grounded-claims policy before responses ship
• Build hallucination-rate dashboards from per-claim scores

Install

npm install @framers/agentos-ext-grounding-guard

Peer dependency: @framers/agentos.

Quickstart

import { AgentOS } from '@framers/agentos';
import { createGroundingGuardrail } from '@framers/agentos-ext-grounding-guard';

const agentos = new AgentOS();
await agentos.initialize({
  extensionManifest: {
    packs: [
      {
        factory: () =>
          createGroundingGuardrail({
            entailmentThreshold: 0.7,
            minGroundedClaims: 0.8,
          }),
        enabled: true,
      },
    ],
  },
});

Public API

• ClaimExtractor — pulls discrete factual claims from a response
• GroundingChecker — runs each claim through NLI scoring
• GroundingGuardrail — guardrail wiring both for the AgentOS contract
• createGroundingGuardrail(options?) — factory returning an ExtensionPack
• createExtensionPack(context) — auto-discoverable factory used by AgentOS extension auto-pickup

See src/types.ts for GroundingGuardOptions.

Examples

• test/ — claim-extraction and entailment scoring fixtures

Lazy loading and optional install

This package is an optional dependency of @framers/agentos-extensions-registry. The registry ships catalog metadata; createCuratedManifest() calls import.meta.resolve() per entry and silently skips anything not installed. npm install @framers/agentos-ext-grounding-guard is the gate.

The NLI entailment pipeline does not load at activation. The pack registers a factory under the grounding:nli-pipeline key in SharedServiceRegistry, and the model file enters the module graph only on the first claim verification. The claim extractor runs first against extracted claims; only claims that pass extraction reach NLI.

GuardrailOutputPayload carries ragSources?: RagRetrievedChunk[] when the response was generated from RAG context. The grounding guardrail uses these as the entailment-source set; without them, the claim extractor still runs but per-claim entailment falls back to whatever passages the host provides.

The guardrail registers with config.evaluateStreamingChunks = false (final-response only) by default, since claim extraction needs full responses. It runs in Phase 2 of the two-phase dispatcher. Below-threshold claims return FLAG by default; configure contradictionAction: 'block' to escalate to BLOCK.

For the full DI model and end-to-end walkthrough, see How extensions stay optional and lazy and the auto-loading guide.

License

Apache 2.0 — see the repo root LICENSE.