clarity-gate

Pre-ingestion verification for epistemic quality in RAG systems.

Core Question: "If another LLM reads this document, will it mistake assumptions for facts?"

Status

⚠️ v0.1.0 is a placeholder release to reserve the package name. Full validation is coming soon.

For working validators now, use the existing packages:

• npm install cgd-validator / pip install cgd-validator
• npm install sot-validator / pip install sot-validator

Or use the Claude skill directly.

What It Does

Clarity Gate validates documents before they enter knowledge bases, ensuring:

• Claims are properly marked as hypotheses vs. established facts
• Projections have uncertainty markers
• Assumptions are explicit, not buried
• Data is internally consistent

Installation

npm install clarity-gate

CLI Usage

# Validate a Clarity-Gated Document
clarity-gate validate-cgd document.cgd

# Validate a Source of Truth file
clarity-gate validate-sot reference.sot

# Auto-detect format
clarity-gate check document.md

# Options
clarity-gate validate-cgd document.cgd --json      # JSON output
clarity-gate validate-cgd document.cgd --strict    # Treat warnings as errors

Programmatic Usage

import { validateCGD, validateSOT, validate } from 'clarity-gate';

const result = await validateCGD('path/to/document.cgd');

if (result.valid) {
  console.log('Document passes all checks');
  console.log('RAG-ingestable:', result.metadata?.ragIngestable);
} else {
  for (const error of result.errors) {
    console.log(`[${error.rule}] ${error.message}`);
  }
}

Document Formats

CGD (Clarity-Gated Document)

Documents that have been verified for epistemic quality:

• YAML frontmatter with verification status
• Epistemic markers inline (e.g., *(not specified)*)
• HITL verification records
• End marker: Clarity Gate: CLEAR | REVIEWED

SOT (Source of Truth)

Authoritative reference documents:

• Strict structure requirements
• Staleness tracking with markers: [STABLE], [CHECK], [VOLATILE]
• Verified claims with citations

Specification

This validator implements:

• CGD_FORMAT.md v1.2 — 24 CGD rules
• SOT_FORMAT.md v1.2 — 7 SOT rules
• VALIDATOR_REFERENCE.md v1.2 — Implementation guide

Related Packages

| Package | Purpose | |---------|---------| | clarity-gate | Unified validator (this package) | | claritygate | Alias → redirects here | | cgd-validator | Legacy CGD validator | | sot-validator | Legacy SOT validator |

Links

• clarity-gate — Main project repository
• arxiparse.org — Live implementation for scientific papers
• LessWrong post — Research writeup

License

CC-BY-4.0