log-narrator

A generic, stateful extraction and detection package that converts heterogeneous operational events into normalized entities, signals, and narratives for explainable higher-order detection.

Install

npm install log-narrator

For Redis-backed production mode (optional):

npm install ioredis

Core Concepts

| Concept | Description | |---|---| | CanonicalEvent | A normalized event mapped from a raw source payload | | Entity | A normalized object (user, file, session, etc.) with external IDs | | Signal | An atomic derived observation (burst access, cross-location, etc.) | | Narrative | A stateful, correlated interpretation built from multiple signals over time | | Finding | A finalized conclusion emitted from a satisfied narrative |

Pipeline Stages

raw event → canonical event → entity extraction → signal detection
         → stateful correlation → narrative rule evaluation → finding

Quick Start

import {
  Pipeline, Registry, MemoryStateStore,
  EventMapper, EntityExtractor, SignalDetector, NarrativeRule,
} from "log-narrator";

// 1. Define an event mapper
const mapper: EventMapper = {
  sourceType: "my-app",
  map(raw) {
    const r = raw as any;
    return {
      id: r.id,
      type: "resource.accessed",
      timestamp: r.timestamp,
      actor: { id: r.userId, type: "user" },
      target: { id: r.resourceId, type: "file" },
      source: "my-app",
      raw: r,
    };
  },
};

// 2. Define entity extractors, signal detectors, narrative rules...

// 3. Build registry
const registry = new Registry()
  .registerMapper(mapper)
  .registerExtractor(myExtractor)
  .registerDetector(myDetector)
  .registerNarrativeRule(myRule);

// 4. Create pipeline (in-memory for dev)
const state = new MemoryStateStore();
const pipeline = new Pipeline(registry, state);

// 5. Process events
const result = await pipeline.process(rawEvent);
console.log(result.signals);     // atomic signals detected
console.log(result.narratives);  // correlated narratives
console.log(result.findings);    // finalized conclusions

// 6. Listen to events
pipeline.on("narrative:satisfied", (n) => console.log("Narrative!", n));
pipeline.on("finding:emitted", (f) => console.log("Finding!", f));

Redis Mode (Production)

import Redis from "ioredis";
import { Pipeline, Registry, RedisStateStore } from "log-narrator";

const redis = new Redis();
const state = new RedisStateStore(redis, "myapp:");
const pipeline = new Pipeline(registry, state);

// Call sweep periodically to expire stale narratives
setInterval(() => pipeline.sweep(), 10_000);

Narrative Rules

Rules declare which signal combinations, over what time window, constitute a narrative:

const rule: NarrativeRule = {
  narrativeType: "likely_ai_driven_access",
  windowSeconds: 30,
  condition: {
    requiredSignals: ["burst_access", "cross_location_access", "missing_navigation"],
    // anyOfSignals, absentSignals, minSignalCount, custom() also supported
  },
  confidence: 0.85,
  buildSummary(signals, entities) {
    return `Detected ${signals.length} signals indicating AI-driven access.`;
  },
  toFinding(narrative) {
    return { /* ... */ };
  },
};

Condition operators

| Field | Meaning | |---|---| | requiredSignals | All must be present (AND) | | anyOfSignals | At least one present (OR) | | absentSignals | None may be present (NOT) | | minSignalCount | Minimum signal count in window | | custom | Async predicate for advanced logic |

Narrative Lifecycle

Narratives progress through: open → accumulating → satisfied / expired / closed / merged

Call pipeline.sweep() periodically to expire stale narratives.

Architecture

┌─────────────┐
│  Raw Events  │
└──────┬──────┘
       │  EventMapper (pluggable, per-source)
       ▼
┌─────────────────┐
│ Canonical Events │
└──────┬──────────┘
       │  EntityExtractor (pluggable)
       ▼
┌──────────┐
│ Entities │
└──────┬───┘
       │  SignalDetector (pluggable, stateful)
       ▼
┌─────────┐
│ Signals │
└──────┬──┘
       │  StateStore (Memory / Redis)
       │  NarrativeRule (pluggable)
       ▼
┌────────────┐     ┌──────────┐
│ Narratives │────▶│ Findings │
└────────────┘     └──────────┘

License

MIT