@credence/ingest
v0.6.0
Published
Credence generic ingestion: map any external source to claims, once. Adapters are thin mappers.
Readme
@credence/ingest
One ingestion mechanism, many thin adapters.
Pulling knowledge from an external source (Graphiti, Mem0, Cognee, a CSV, a SQL row, a document extractor, an API or tool response) splits into two very different jobs:
- Mechanism — resolve the subject entity, attach the source as evidence, record the claim idempotently, keep the status honest. Identical for every source.
- Mapping — "their
HAS_DEBTfield is mytotal_debtkey." Specific to each source.
@credence/ingest puts the mechanism in one place (importFacts) so an adapter is
just a pure function (theirRecord) => ExternalFact[] — never a re-implementation.
import { importFacts, type ExternalFact } from "@credence/ingest";
const facts: ExternalFact[] = records.map((r) => ({
subject: { type: "company", label: r.name, identifier: r.id },
key: "total_debt",
value: r.debt,
source: { kind: "document", uri: r.docUrl }, // provenance → evidence
// status defaults to "reported" — imports are hearsay until verified
}));
const result = await importFacts(ledger, caseId, facts);
// → { imported, created, skipped, claimRefs }
await rules.recompute(caseId); // now Credence tells you what's still MISSINGHonesty guard
importFacts refuses to mint verified (or disputed) claims. Everything comes
in as reported (or inferred), with the source attached as evidence. A value having
a source is provenance, not verification — the manifesto's first rule. A later
verification pass (or @credence/judge) upgrades a claim once it checks out against a
primary source. Two imported sources that disagree simply become a disputed finding
via the rules engine — nothing is overwritten or averaged.
Six reference mappers, five very different shapes
All of them end in the same importFacts(ledger, caseId, facts) call.
1. Graph edges — Graphiti (and any temporal knowledge graph):
import { graphitiFacts, importFacts } from "@credence/ingest";
const facts = graphitiFacts(await graphiti.search({ group_id: caseId }), {
keyMap: { HAS_DEBT: "total_debt", REGISTERED_AS: "cnpj" },
});
await importFacts(ledger, caseId, facts);2. Table rows — CSV, spreadsheet, SQL view. Here the mapping is pure configuration, not code:
import { tabularFacts } from "@credence/ingest";
const facts = tabularFacts(csvRows, {
subject: { type: "company", labelColumn: "Company", identifierColumn: "Reg" },
columns: { "Razao Social": "legal_name", Debt: "total_debt" },
transform: { Debt: (v) => Number(v) },
source: { kind: "document", uri: "portfolio.csv" },
});An empty cell is skipped, never recorded as null — a blank means unknown
(nobody captured it), and collapsing that into a value is precisely the confident
failure Credence exists to prevent. Use ledger.recordAbsence() when you actually
checked and found nothing.
3. Extracted document fields — Docling / Reducto / LandingAI. This is the one that delivers "audit the number down to the pixel":
import { extractionFacts } from "@credence/ingest";
const facts = extractionFacts(await docling.extract(pdf), {
keyMap: { razao_social: "legal_name", divida_total: "total_debt" },
subject: { type: "company", label: "Acme Inc.", identifier: "REG-1" },
});
// page + bbox flow straight into each claim's evidence locator,
// and confidence < 0.5 is imported as `inferred` instead of `reported`.4. Agent-memory systems — Mem0 and Cognee:
import { mem0Facts, cogneeFacts } from "@credence/ingest";
const facts = mem0Facts(memories, { keyMap: { timezone: "timezone" } });A caveat stated plainly in the code: these systems mostly hold narrative memory
("the user prefers morning meetings"). That is Mem0's lane and Credence does not
compete with it. Only records carrying structure — a metadata field, a graph
triple — become claims. Free text is skipped rather than stuffed into a value,
because a sentence in a value field is not an auditable assertion and pretending
otherwise would corrupt the ledger. mem0Facts therefore requires a keyMap:
without one there is nothing structured to lift.
5. A tool or API response — the most common thing an agent has to justify itself with:
import { apiFacts } from "@credence/ingest";
const facts = apiFacts(await bureau.get(`/companies/${reg}`), {
subject: { type: "company", labelPath: "$.company.name", identifierPath: "$.company.registration" },
paths: { "$.company.debts[0].amount": "total_debt", "$.company.registration": "cnpj" },
source: {
uri: `bureau://v2/companies/${reg}`,
meta: { provider: "bureau", requestHash, receivedAt },
},
});The convention it follows — kind: "tool_receipt", uri: "<provider>://<endpoint>",
meta: { requestHash, provider, receivedAt } — is documented in
docs/SPEC.md, and @credence/otel's traceEvidence
already does the same for spans.
The response path becomes the locator. Without it a receipt says "the bureau
said so"; with it, a reviewer can go to the exact field — the tool-output
equivalent of a page and a bounding box. An absent field is skipped for the same
reason an empty cell is: the provider having nothing is unknown, never a value.
Writing "an adapter for the other one too" means writing another mapper of this size — a database view, a registry endpoint — never another pipeline.
