# @exellix/narrix-ingest

Thin **CNI producer hub** (registry + router) that converts raw inputs (starting with **records/JSON**) into **CNI v1.1** by delegating to registered adapters.

`narrix-ingest` **does not run Narrix**. It only returns `CniV11` + diagnostics.  
Execution happens later via `@exellix/narrix-runner` (pack resolution + engine).

---

## Why this package exists

We keep strict single-responsibility boundaries:

- **Adapters**: source-shape → **CNI v1.1**
  - `@exellix/narrix-adapter-records` (JSON records)
  - (later) text/chat/docs adapters
- **Runner**: **CNI + datasetId** → stories/signals (pack + engine)

`narrix-ingest` sits **above adapters**, so callers don’t implement adapter selection and don’t grow into a “mega-ingestion module”.

---

## Architecture

raw input (records/text/chat/docs) → @exellix/narrix-ingest (pick producer + call it) → CniV11 → @exellix/narrix-runner (resolve pack + run engine) → { stories, signals }

---

## Install

```bash
npm i @exellix/narrix-ingest

Dependencies

• Runtime dependency: @exellix/narrix-cni (types)
• No adapter dependencies by default (keeps ingest tiny)

Adapters are registered by the caller:

npm i @exellix/narrix-adapter-records

Quick start (records → CNI → runner)

import { createIngest } from "@exellix/narrix-ingest";
import { recordAdapter } from "@exellix/narrix-adapter-records";
import { createRunner } from "@exellix/narrix-runner";

// 1) Create ingest and register producers (adapters)
const ingest = createIngest();
ingest.registry.register(recordAdapter, { makeDefault: true });

// 2) Create runner once (stateless, reuse)
const runner = await createRunner();

// 3) Convert a record into CNI
const { cni, producer, diagnostics } = ingest.toCni({
  kind: "records",
  input: {
    record: { assetId: "host-01", hostname: "web-01.prod", ip: "10.0.1.50" },
    recordType: "asset",
    schema: { idField: "assetId", displayNameField: "hostname" }
  }
});

// 4) Run Narrix (pack resolution + engine)
const out = await runner.run({
  cni,
  datasetId: "neo.vulnerabilities"
});

console.log(producer);     // { kind, adapterId, version }
console.log(out.stories);
console.log(out.signals);

Core concepts

Producer (adapter) contract

A producer is anything that exposes:

• kind — "records" | "text" | "chat" | "docs" | string
• adapterId — stable identity (e.g. narrix.adapter.record.v1)
• version — semver
• toCni(input, options?) — returns { cni, diagnostics? }

This is intentionally duck-typed so ingest can work with all Narrix adapters without hard dependencies.

API

createIngest(options?)

import { createIngest } from "@exellix/narrix-ingest";

const ingest = createIngest({
  runtime: {
    onMissingProducer: "throw",     // "throw" | "returnError" (default: "throw")
    deterministicSort: true         // default: true
  }
});

Returns an ingest instance with:

• ingest.registry — registry for producers
• ingest.toCni(req) — convert one input to CNI
• ingest.toCniMany(req) — convert many inputs to CNI (batch)

Registry

ingest.registry.register(producer, opts?)

ingest.registry.register(recordAdapter, { makeDefault: true });

Options:

• makeDefault?: boolean — make this producer the default for its kind

Rules:

• Duplicate (kind, adapterId) registration throws INGEST_DUPLICATE_PRODUCER
• Setting default when a default exists throws (safe-by-default)

ingest.registry.get(kind, adapterId?)

const p = ingest.registry.get("records");                 // default records producer
const p2 = ingest.registry.get("records", "some.id.v2");  // explicit producer

ingest.registry.list()

Returns metadata for diagnostics and observability:

[
  { kind: "records", adapterId: "narrix.adapter.record.v1", version: "1.0.0", isDefault: true }
]

Adding a new adapter

When a new adapter package is published:

1. Add it to dependencies in package.json: "@exellix/narrix-adapter-<kind>": "^x.y.z"

2. Import it in src/defaultIngest.ts: import { <kind>Adapter } from "@exellix/narrix-adapter-<kind>"; (or import { adapter as <kind>Adapter } from ... if the package exports adapter)

3. Register it in the IIFE: instance.registry.register(<kind>Adapter, { makeDefault: true });

4. Update the JSDoc comment in defaultIngest.ts — move from "Pending" to "Current defaults".

5. Bump version and publish.

Currently pending (not yet registered in defaultIngest):

• @exellix/narrix-adapter-text
• @exellix/narrix-adapter-chat
• @exellix/narrix-adapter-docs

ingest.toCni(req)

const res = ingest.toCni({
  kind: "records",
  input: RecordAdapterInput,
  adapterId?: "narrix.adapter.record.v1",  // optional override
  options?: unknown                        // forwarded to producer
});

Response:

type ToCniResponse = {
  cni: CniV11;
  diagnostics?: unknown;
  producer: {
    kind: string;
    adapterId: string;
    version: string;
  };
  error?: never;
};

type ToCniErrorResponse = {
  error: true;
  code: IngestErrorCode;
  message: string;
  kind?: string;
  adapterId?: string;
};

If runtime.onMissingProducer === "returnError", missing producers return a ToCniErrorResponse. Otherwise, ingest throws IngestError.

Batch: ingest.toCniMany(req)

const batch = ingest.toCniMany({
  inputs: [
    { kind: "records", input: rec1 },
    { kind: "records", input: rec2 }
  ],
  onError: "attachError"  // "throw" | "attachError" | "skip" (default: "attachError")
});

console.log(batch.meta);       // { total, succeeded, failed, skipped }
console.log(batch.results);    // (ToCniResponse | ToCniErrorResponse)[]

Error codes

| Code | Meaning | | --------------------------- | ---------------------------------------------------------------- | | INGEST_PRODUCER_NOT_FOUND | No default producer for kind, and no (kind, adapterId) match | | INGEST_PRODUCER_FAILED | Producer threw an error | | INGEST_INVALID_INPUT | Input missing/invalid for the requested kind (basic guards) | | INGEST_DUPLICATE_PRODUCER | Producer (kind, adapterId) already registered |

All thrown errors are IngestError extends Error with:

• code, message, optional cause, optional details

Determinism guarantees

• narrix-ingest itself does not add timestamps or random IDs.
• It forwards options to producers; producers (adapters) are expected to be deterministic.
• Any sorting performed by ingest (if enabled) must be stable.

“Smarter over time” (without breaking v1)

toCni({ kind, input }) remains the stable, explicit API.

Future additive APIs (optional):

• detectKind(input, hints?) — deterministic heuristics to suggest "records" | "text" | ..."
• router(input, hints?) — caller-provided routing hook returning { kind, adapterId, options }
• Profiles: named routing decisions to avoid repeating config

These additions do not change runner or adapter contracts.

Package relationships

@woroces/ai-tasks
  ├── @exellix/narrix-ingest
  │       └── (registered) @exellix/narrix-adapter-records  (+ future adapters)
  └── @exellix/narrix-runner
          ├── @exellix/narrix-packs-library
          └── @exellix/narrix-engine

Publishing

This package is published as a private scoped package to GitHub Packages.

1. Ensure .npmrc in the repo (or your user directory) contains the GitHub Packages registry and auth token for @exellix:
   • @exellix:registry=https://npm.pkg.github.com
   • //npm.pkg.github.com/:_authToken=YOUR_TOKEN (do not commit the token; use env or local .npmrc only)
2. Build and publish:

   npm run build
   npm publish

   package.json has "publishConfig": { "registry": "https://npm.pkg.github.com" }, so npm publish uses the token from .npmrc automatically.

Auth

Uses the token in your repo's .npmrc / user .npmrc. Do not paste tokens into code or docs.

License

Proprietary (internal).