@semiont/make-meaning

Making meaning from resources through actors, context assembly, and relationship reasoning.

This package implements the actor model from ACTOR-MODEL.md. It owns the Knowledge Base and the actors that interface with it:

• Stower (write) — the single write gateway to the Knowledge Base; handles all resource and annotation mutations and job lifecycle events
• Browser (read) — handles all KB read queries: resources, annotations, events, annotation history, referenced-by lookups, entity type listing, and directory browse (merging filesystem listings with KB metadata)
• Gatherer (context assembly) — assembles gathered context for annotations (gather:requested) and resources (gather:resource-requested); searches vectors for semantically similar passages (adds semanticContext to GatheredContext)
• Matcher (search/link) — context-driven candidate search with multi-source retrieval, composite structural scoring, and optional LLM semantic scoring
• Smelter (embed) — subscribes to resource/annotation events, chunks text, embeds via @semiont/vectors, emits embedding:compute commands (persisted by Stower as embedding:computed events), and indexes into vector store (Qdrant)
• CloneTokenManager (yield) — manages clone token lifecycle for resource cloning

All actors subscribe to the EventBus via RxJS pipelines. They expose only initialize() and stop() — no public business methods. Callers communicate with actors by putting events on the bus.

The EventBus is a complete interface for all knowledge-domain operations. HTTP routes in the backend are thin wrappers that delegate to EventBus actors. The @semiont/api-client exposes the same operations via verb-oriented namespaces (semiont.browse, semiont.mark, semiont.gather, etc.).

Quick Start

npm install @semiont/make-meaning

Start Make-Meaning Service

import { startMakeMeaning } from '@semiont/make-meaning';
import { SemiontProject } from '@semiont/core/node';
import { EventBus } from '@semiont/core';
import type { Logger } from '@semiont/core';

// EventBus is created outside make-meaning — it is not encapsulated by this package
const eventBus = new EventBus();
const project = new SemiontProject('/path/to/project');

// Start all infrastructure
const makeMeaning = await startMakeMeaning(project, config, eventBus, logger);

// Access components
const { knowledgeSystem, jobQueue } = makeMeaning;
const { kb, stower, browser, gatherer, matcher, smelter, cloneTokenManager } = knowledgeSystem;

// Graceful shutdown
await makeMeaning.stop();

This single call initializes:

• KnowledgeSystem — groups the Knowledge Base and its actors
  • KnowledgeBase — groups EventStore, ViewStorage, WorkingTreeStore, GraphDatabase, GraphDBConsumer, and optionally VectorStore and Smelter
  • Stower — subscribes to write commands on EventBus
  • Browser — subscribes to all KB read queries and directory browse requests on EventBus
  • Gatherer — subscribes to annotation and resource gather requests on EventBus; searches vectors for semantically similar passages
  • Matcher — subscribes to candidate search requests on EventBus
  • Smelter — subscribes to resource/annotation events, chunks text, embeds, indexes into Qdrant
  • CloneTokenManager — subscribes to clone token operations on EventBus
• JobQueue — background job processing queue + job status subscription
• 6 annotation workers — poll job queue for async AI tasks

Gather Context (via EventBus)

import { firstValueFrom, race, filter, timeout } from 'rxjs';

// Emit gather request for an annotation
eventBus.get('gather:requested').next({
  annotationUri,
  resourceId,
  options: { contextLines: 5 },
});

// Await result
const result = await firstValueFrom(
  race(
    eventBus.get('gather:complete').pipe(filter(e => e.annotationUri === annotationUri)),
    eventBus.get('gather:failed').pipe(filter(e => e.annotationUri === annotationUri)),
  ).pipe(timeout(30_000)),
);

Architecture

Actor Model

All meaningful actions flow through the EventBus. The KB actors are reactive — they subscribe via RxJS pipelines in initialize() and communicate results by emitting on the bus.

graph TB
    Routes["Backend Routes"] -->|commands| BUS["Event Bus"]
    Workers["Job Workers"] -->|commands| BUS
    EBC["SemiontApiClient"] -->|commands| BUS

    subgraph ks ["Knowledge System"]
        STOWER["Stower<br/>(write)"]
        BROWSER["Browser<br/>(read)"]
        GATHERER["Gatherer<br/>(context assembly)"]
        MATCHER["Matcher<br/>(search/link)"]
        SMELTER["Smelter<br/>(embed)"]
        CTM["CloneTokenManager<br/>(clone)"]
        KB["Knowledge Base"]
        VECTORS["Vector Store<br/>(Qdrant)"]
        STOWER -->|persist| KB
        BROWSER -->|query| KB
        GATHERER -->|query| KB
        GATHERER -->|search| VECTORS
        MATCHER -->|query| KB
        MATCHER -->|search| VECTORS
        SMELTER -->|embed & index| VECTORS
        SMELTER -->|read| KB
        CTM -->|query| KB
    end

    BUS -->|"yield:create, yield:update, yield:mv<br/>mark:create, mark:delete, mark:update-body<br/>frame:add-entity-type, mark:archive, mark:unarchive<br/>mark:update-entity-types, job:start, job:*"| STOWER
    BUS -->|"browse:resource-requested, browse:resources-requested<br/>browse:annotations-requested, browse:annotation-requested<br/>browse:events-requested, browse:annotation-history-requested<br/>browse:referenced-by-requested, browse:entity-types-requested<br/>browse:directory-requested"| BROWSER
    BUS -->|"gather:requested<br/>gather:resource-requested"| GATHERER
    BUS -->|"match:search-requested"| MATCHER
    BUS -->|"yield:created, mark:created,<br/>mark:body-updated"| SMELTER
    BUS -->|"yield:clone-token-requested<br/>yield:clone-resource-requested<br/>yield:clone-create"| CTM

    STOWER -->|"yield:created, yield:updated, yield:moved<br/>mark:created, mark:deleted, mark:body-updated<br/>frame:entity-type-added, ..."| BUS
    BROWSER -->|"browse:resource-result, browse:resources-result<br/>browse:annotations-result, browse:annotation-result<br/>browse:events-result, browse:annotation-history-result<br/>browse:referenced-by-result, browse:entity-types-result<br/>browse:directory-result"| BUS
    GATHERER -->|"gather:complete, gather:failed<br/>gather:resource-complete, gather:resource-failed"| BUS
    MATCHER -->|"match:search-results, match:search-failed"| BUS
    SMELTER -->|"embedding:compute,<br/>embedding:delete"| BUS
    CTM -->|"yield:clone-token-generated<br/>yield:clone-resource-result<br/>yield:clone-created"| BUS

    classDef bus fill:#e8a838,stroke:#b07818,stroke-width:3px,color:#000,font-weight:bold
    classDef actor fill:#5a9a6a,stroke:#3d6644,stroke-width:2px,color:#fff
    classDef kb fill:#8b6b9d,stroke:#6b4a7a,stroke-width:2px,color:#fff
    classDef caller fill:#4a90a4,stroke:#2c5f7a,stroke-width:2px,color:#fff

    class BUS bus
    classDef vectorstore fill:#6b8e9d,stroke:#4a6a7a,stroke-width:2px,color:#fff
    class STOWER,BROWSER,GATHERER,MATCHER,SMELTER,CTM actor
    class KB kb
    class VECTORS vectorstore
    class Routes,Workers,EBC caller

Knowledge System and Knowledge Base

The Knowledge System binds the Knowledge Base to its actors. Nothing outside the Knowledge System reads or writes the Knowledge Base directly.

The Knowledge Base is an inert store — it has no intelligence, no goals, no decisions. It groups five core subsystems and two optional ones:

| Store | Implementation | Purpose | |-------|---------------|---------| | Event Log | EventStore | Immutable append-only log of all domain events | | Materialized Views | ViewStorage | Denormalized projections for fast reads | | Content Store | WorkingTreeStore | Working-tree files addressed by URI | | Graph | GraphDatabase | Eventually consistent relationship projection | | Graph Consumer | GraphDBConsumer | Event-to-graph synchronization pipeline | | Vectors (optional) | VectorStore | Semantic vector index (Qdrant + memory) via @semiont/vectors | | Smelter (optional) | Smelter | Embedding pipeline actor (chunk, embed, index) |

import { createKnowledgeBase } from '@semiont/make-meaning';

const kb = await createKnowledgeBase(eventStore, project, graphDb, logger);
// kb.eventStore, kb.views, kb.content, kb.graph, kb.graphConsumer
// kb.vectors (optional), kb.smelter (optional)

EventBus Ownership

The EventBus is created by the backend (or script) and passed into startMakeMeaning() as a dependency. Make-meaning does not own or encapsulate the EventBus — it is shared across the entire system.

Pure projection validators

The dispatcher in src/handlers/job-commands.ts does projection-validated job creation: when a mark.assist (linking) or yield.fromAnnotation job arrives with entityTypes, the dispatcher validates that every tag is registered; when a tagging job arrives with a schemaId, the dispatcher resolves it against the registered tag-schema set.

Both rules are pure functions in src/views/projection-validators.ts:

• resolveTagSchema(schemas, schemaId) → { schema } | { error } — id lookup with the standard "Tag schema not registered" / "tag-annotation requires schemaId" error formats.
• validateEntityTypes(registered, requested) → { ok: true } | { ok: false; unknown } — set membership check that lists the offending tags in caller order.

The dispatcher is the I/O shell: read the projection (via the readers in src/views/), pass it to the validator (pure), then either stash the resolved value or rethrow as job:create-failed. Validator unit tests run in single-digit milliseconds with no filesystem, no event-bus, no mock JobQueue — the dispatcher integration tests in __tests__/handlers/job-commands.test.ts keep the wiring covered.

This pattern (functional core, imperative shell) is shared with @semiont/event-sourcing's projection reducers; see docs/system/PROJECTION-PATTERN.md for the architectural narrative, the full axiom catalog, and guidance for adding new validators.

Documentation

• Architecture — Actor model, data flow, storage architecture
• API Reference — Context modules and operations
• Examples — Common use cases and patterns
• Job Workers — Async annotation workers (in @semiont/jobs)
• Scripting — Direct scripting without HTTP backend

Exports

Service (Primary)

• startMakeMeaning(project, config, eventBus, logger) — Initialize all infrastructure
• MakeMeaningService — Type for service return value (knowledgeSystem, jobQueue, workers, stop)

Knowledge System

• KnowledgeSystem — Interface grouping the Knowledge Base and its actors
• stopKnowledgeSystem(ks) — Ordered teardown of the Knowledge System

Knowledge Base

• createKnowledgeBase(eventStore, project, graphDb, logger) — Async factory function
• KnowledgeBase — Interface grouping the five KB stores (including graphConsumer)

Actors

• Stower — Write gateway actor
• Browser — Read actor (all KB queries, directory listings merged with KB metadata)
• Gatherer — Context assembly actor (annotation and resource gather flows; vector semantic search)
• Matcher — Search/link actor (context-driven candidate search with structural + semantic scoring)
• Smelter — Embedding pipeline actor (chunk, embed, persist, index into vector store)
• CloneTokenManager — Clone token lifecycle actor (yield domain)

Operations

• ResourceOperations — Resource CRUD (emits commands to EventBus)
• AnnotationOperations — Annotation CRUD (emits commands to EventBus)

Context Assembly

• ResourceContext — Resource metadata queries from ViewStorage
• AnnotationContext — Annotation queries and LLM context building
• GraphContext — Graph traversal and search
• LLMContext — Resource-level LLM context assembly

Generation

• generateResourceSummary — Resource summarization
• generateReferenceSuggestions — Smart suggestion generation

Dependencies

• @semiont/core — Core types, EventBus, utilities
• @semiont/api-client — OpenAPI-generated types
• @semiont/event-sourcing — Event store and view storage
• @semiont/content — Content-addressed storage
• @semiont/graph — Graph database abstraction
• @semiont/ontology — Schema definitions for tags
• @semiont/inference — AI primitives (generateText)
• @semiont/vectors — Vector store abstraction (Qdrant + memory) and embedding providers (Voyage, Ollama)
• @semiont/jobs — Job queue and annotation workers

Testing

npm test                # Run tests
npm run test:watch      # Watch mode
npm run test:coverage   # Coverage report

License

Apache-2.0