@orchestree/memory-shared

Shared memory-tier primitives for the Orchestree vault system. Consumable by both @orchestree/api (this monorepo) and the external memory-api service (orchestree-memory-build/api) once published.

Purpose

The vault subsystem is being split across two repos: a Kubernetes-deployed memory-api service that owns GCS / NATS / Redis on the cluster, and the in-monorepo @orchestree/api Express service that owns user-facing routes. Both repos need the same low-level primitives — chunking, lexical search, write-side cache+NATS+audit fan-out, search-result hydration, semantic-cache invalidation. This package is that shared surface, with zero direct dependency on either host's logger or DB-singleton modules.

Each module corresponds to an open question tracked in docs/vault-system/21-open-questions.md:

| Module | OQ | Resolves | | ------------------------------- | ----- | -------------------------------------------------------------- | | chunker.ts | OQ-13 | Paragraph/sentence-aware text splitter for vault_chunks rows | | vaultIndexer.ts | OQ-13 | Chunked-embedding UPSERT + prune against vault_chunks | | vaultSearch.ts | OQ-14 | GIN-indexed ts_vector lexical search | | vaultWriter.ts | OQ-16 | Post-GCS side-effect wrappers (try/catch around the seven ops) | | searchHydrator.ts | OQ-17 | Redis-first / GCS-fallback / write-through note hydration | | semanticCacheInvalidator.ts | OQ-18 | Per-tenant version-bump cache invalidation |

Public API

import {
  // Logger contract
  type Logger,
  noopLogger,

  // OQ-13
  chunkText,
  indexNoteChunks,
  type Queryable,            // pg.Pool | pg.PoolClient
  type EmbedBatch,
  type IndexNoteChunksOptions,
  type IndexNoteChunksResult,

  // OQ-14
  vaultLexicalSearch,
  type VaultSearchOptions,
  type VaultSearchRow,

  // OQ-16
  runPostGcsWriteSideEffects,
  runPostGcsUpdateSideEffects,
  runPostGcsDeleteSideEffects,
  tenantKey,
  topicVaultWrite,
  topicVaultUpdate,
  topicVaultDelete,
  type WriteNoteParams,
  type UpdateNoteParams,
  type DeleteNoteParams,
  type CacheClient,
  type NatsClient,
  type AuditLogger,
  type QuotaCounter,

  // OQ-17
  hydrateSearchResults,
  noteCacheKey,
  type RedisLike,
  type GcsNoteReader,
  type NoteRef,

  // OQ-18
  buildSearchCacheKey,
  getTenantSearchVersion,
  bumpTenantSearchVersion,
  resetTenantSearchVersion,
  type RedisInvalidatorClient,
} from '@orchestree/memory-shared'

Logger injection

The package never imports a host application's logger. Every function that emits structured log lines either takes a Logger parameter directly or accepts one on its options object:

export interface Logger {
  info(message: string, meta?: Record<string, unknown>): void
  warn(message: string, meta?: Record<string, unknown>): void
  error(message: string, meta?: Record<string, unknown>): void
}

apps/api/src/lib/logger.ts satisfies this directly. Pino, winston, and bare console are trivially adapted. If you pass nothing the package falls back to noopLogger (silent).

Database / Redis injection

The package does not call getPool() or getRedis(). Callers pass:

• a Queryable (any pg.Pool or pg.PoolClient) into indexNoteChunks
• a pg.Pool into vaultLexicalSearch
• a RedisLike (mget + pipeline) into hydrateSearchResults
• a RedisInvalidatorClient (get + set + incr) into the cache-invalidator fns
• CacheClient / NatsClient / AuditLogger / QuotaCounter interfaces into the vaultWriter post-GCS fan-out functions

pg and ioredis are declared as peerDependencies so the package contributes zero runtime to consumers that already ship them.

Consumer wiring

In this monorepo (@orchestree/api)

Add the workspace dependency to apps/api/package.json:

"dependencies": {
  "@orchestree/memory-shared": "workspace:*"
}

Then either import from the package root directly, or use the thin adapter at apps/api/src/ai/memory/sharedAdapter.ts which wires the local logger from apps/api/src/lib/logger.ts so call sites stay terse.

In the external memory-api repo

Once this package is published to the registry:

pnpm add @orchestree/memory-shared

The external repo wires its own logger (pino) and its own GCS / Redis / NATS clients into the exported functions.

Versioning

Semver, starting at 0.1.0. Breaking changes to any exported type or function signature bump the minor (pre-1.0) or major (post-1.0) per semver rules.

The OQ-N stubs in this package are reference implementations that the external memory-api repo will consume verbatim once it imports @orchestree/memory-shared. Any change to a public signature here implies an update to the memory-api caller in lockstep — coordinate before bumping.

Publishing

The package is currently flagged private: true to prevent accidental npm publication. To publish:

1. Bump version in package.json per semver.
2. Remove "private": true (or pass --access public to npm publish).
3. Run pnpm --filter @orchestree/memory-shared build.
4. pnpm --filter @orchestree/memory-shared publish (the prepublishOnly hook re-runs the build).

The files field restricts the tarball to dist/, so consumers receive only the compiled .js + .d.ts + sourcemaps.

Cross-references

• docs/vault-system/21-open-questions.md §OQ-13, OQ-14, OQ-16, OQ-17, OQ-18
• docs/vault-system/03-data-model.md §2.5–2.6 (vault_notes, vault_chunks)
• docs/vault-system/07-memory-flow.md §2–5, §9, §11 (write/update/delete flows)
• docs/vault-system/11-api-reference.md §POST /v1/vault/search
• docs/vault-system/17-implementation-plan.md (memory-api consolidation plan)