@gonk/memory

Three-layer memory substrate: curated markdown files, SQLite session transcripts with vector recall, and a structured key-value store. Plus a temporal knowledge graph, per-op cost accounting, and a consolidation pass for deduplication and cold compression.

See docs/memory-design.md for the full design.

Layers

curated   — markdown files per tier: <tier-home>/memory/curated.md, user.md
session   — SQLite per tier: turns indexed via FTS5 + sqlite-vec embeddings
kv        — SQLite per tier: namespaced key/value store

The Memory facade composes all three. createMemory(opts, scopeEnv) is the standard entry point.

Scoring

scoring.ts is pure and side-effect-free. composeScore multiplies five axes:

| axis | function | notes | |---|---|---| | base | — | FTS rank or cosine similarity | | recency | recencyDecay | exponential half-life; Infinity = off | | importance | importanceClamp | stored per-turn ∈ [0, 1] | | veracity | veracityWeight | stated 1.0 → tool 0.5 | | tier | tierWeight | hot 1.0, warm 0.5, cold 0.25 |

Pass scoring: {} to recall() to enable importance + veracity. Add recencyHalflifeHours or tier thresholds to enable the other axes.

Recall fusion

recall() runs one of two strategies, selected by fusion:

| fusion | behavior | |---|---| | "single" (default) | one voice — vector cosine when the embedder is healthy, FTS5 keyword when it is not. Legacy ordering, unchanged. | | "rrf" | every available voice in parallel (vector + FTS5, or FTS5 alone when the embedder is down), fused by Reciprocal Rank Fusion. |

RRF (rrf.ts, pure and side-effect-free) discards each voice's raw score and keeps only an entry's rank within that voice. The fused score is Σ 1/(k + rank) over the voices that returned the entry, with k = 60 (the canonical Cormack et al. constant; also what mnemopi uses). Because the 1/(k+rank) curve is steep at the top and flat in the tail, an entry that lands mid-pack in both voices can overtake one that tops a single voice — the point of polyphonic recall. Vector cosine and negated bm25 are incommensurable, so rank fusion is more robust than weighting their raw scores.

const hits = await handle.recall("kettle on the stove", { fusion: "rrf" });

• Each organ is queried once per tier — candidates are reused, not re-queried.
• Dedup is by entry identity (scope:id): an entry surfaced by both voices fuses into one row whose score is its RRF score.
• The embedder-down path degrades to FTS-only fusion (one voice), preserving the same graceful-degradation contract as "single"; results carry recallMode: "fts".
• Combine with scoring — when both are set, the fused RRF score becomes the base axis the multiplicative composition reranks on.

The fusion primitive is exported directly for reuse:

import { reciprocalRankFusion, RRF_K } from "@gonk/memory";
const fused = reciprocalRankFusion([voiceA, voiceB]); // each: { key, score, item }[]

Additional concerns

• TriplesLayer — temporal SPO knowledge graph backed by SQLite. assert, query, invalidate. Point-in-time queries via asOf.
• checkTriplesFreshness (triple-freshness.ts) — recall-path provenance for the triple store. An anchor-bearing triple (subject = repo, predicate = code-anchor, object = symbol/file) is a cache of external reality; this greps each anchor against the live repo and surfaces stale / repo-missing ones rather than letting them load silently. Off by default and cost-gated: only anchor triples are probed (ordinary ones pay nothing), each (repo, anchor) is grepped at most once per call, and a per-call budget caps distinct probes — past it, anchors come back skipped-budget (honest about not checking). Like the skill probe, it catches a symbol that is gone, not behaviour that drifted. isTripleStale flags stale/repo-missing (never skipped-budget).
• CostLog — records per-op timings and token estimates. totals() aggregates by operation.
• consolidateMemory — deduplicates curated entries and compresses cold session turns via an auxiliary LLM pass.
• scanMemoryContent — optional threat scan before writes; ThreatBlockedError on match.

Storage layout

<tier-home>/memory/curated.md
<tier-home>/memory/user.md
<tier-home>/memory/sessions.db     (FTS5 + turns + session metadata)
<tier-home>/memory/embeddings.db   (sqlite-vec vectors)
<tier-home>/memory/kv.db

Entry points

import { createMemory, MemoryImpl } from "@gonk/memory";
import type { Memory, SessionHandle, SearchResult } from "@gonk/memory/types";
import { FileCuratedMemoryLayer } from "@gonk/memory/curated";
import { SqliteSessionLayer } from "@gonk/memory/session";
import { SqliteKvLayer } from "@gonk/memory/kv";
import { scanMemoryContent } from "@gonk/memory/threat-scan";
import { composeScore, recencyDecay, veracityWeight } from "@gonk/memory";
import { TriplesLayer, CostLog, consolidateMemory } from "@gonk/memory";

Quick example

import { createMemory } from "@gonk/memory";
import { NoopEmbeddingProvider } from "@gonk/embedding";

const memory = createMemory(
  { scope, embedding: new NoopEmbeddingProvider() },
  scopeEnv,
);

const handle = memory.session.open({ sessionId: "abc-123" });
await handle.append({ ts: Date.now(), role: "user", content: "hello" });
const hits = await handle.recall("hello", { limit: 5 });
await handle.close();
await memory.close();