@soulcraft/cor
v3.0.17
Published
Native Rust acceleration for Brainy — SIMD distance, vector quantization, zero-copy mmap, native embeddings. Free tier for storage, Pro license for compute acceleration.
Downloads
1,560
Maintainers
Readme
Brainy is the open-source knowledge database — vectors, relationships, metadata, and full-text in one MIT-licensed engine. Cor swaps compiled Rust in under every hot path. No code changes. No configuration. Install it and brainy (≥ 8.0.9) auto-detects it — loudly: activation prints Providers: 10/10 native, a missing license warns by name, and a broken install makes init() throw rather than quietly run slow. Remove it and everything keeps working on the same files, at open-source speed. That's the contract in both directions.
| Brainy runs it as | Cor replaces it with | You get |
|---|---|---|
| HNSW vector index (in-RAM) | Adaptive DiskANN — mmap-native, self-tuning, streaming segments | billion-scale ANN on one box; recall@10 0.96 at 100M (measured) |
| find() composed in JS | one fused native query: metadata ∩ graph ∩ vector | exact filters inside the vector walk, not after it |
| JSON field index | LSM postings + columnar sort/aggregate, mmap-durable | cold open serves instantly — restart = warm (measured) |
| graph adjacency in JS maps | u64 LSM adjacency + native traversal/analytics | whole-graph reads as single cursor walks |
| JS distance loops | SIMD kernels (f32, SQ8, SQ4-packed) | the hot inner loop in vector search, compiled |
| WASM embedding model | Candle ML runtime, in-process | batch on-box embeddings — no API fees, no data egress |
| time-travel reads in JS | generation-pinned reads across all three substrates | asOf() at native speed |
Quick start
npm install @soulcraft/cor # or: bun add @soulcraft/corimport { Brainy, NounType, VerbType } from '@soulcraft/brainy'
const brain = new Brainy({ storage: { type: 'filesystem', path: './data' } })
await brain.init() // cor auto-detected (brainy ≥ 8.0.9) — look for "Providers: 10/10 native"
const cor = await brain.add({
data: 'Cor swaps compiled Rust in under every Brainy hot path',
type: NounType.Concept,
subtype: 'library',
metadata: { layer: 'native', year: 2026 },
})
const brainy = await brain.add({
data: 'Brainy is the open-source knowledge database',
type: NounType.Concept,
subtype: 'library',
metadata: { layer: 'engine', year: 2026 },
})
await brain.relate({ from: cor, to: brainy, type: VerbType.DependsOn, subtype: 'runtime' })
const hits = await brain.find({
query: 'native acceleration', // vector — what it means
where: { layer: 'native' }, // metadata — pushed INTO the walk
connected: { to: brainy }, // graph — what it touches
}) // one call, three indexes, fusedGet a license & activate — under 1M entities, skip this: brainy alone is free and genuinely enough. Beyond that, pick a tier at soulcraft.com/pricing (pricing follows the size of your brain — from $49/mo), then:
npx @soulcraft/cor login # browser sign-in, like `gh auth login` —
# walks you through checkout too if you don't have a key yet
# or, for servers / CI: export COR_LICENSE_KEY=sc_cor_...Keys verify offline in Rust — no network call at startup, no license server to run. And a missing or expired key never breaks anything: cor steps aside and you're on open-source brainy, same files.
Exact filters inside the vector walk
Pure vector databases post-filter — search first, discard non-matches — so selective filters return too few or lower-recall results. Cor computes the exact matching set first (roaring-bitmap intersection), searches only that provably-correct space, and switches to an exact scan when the set is small (recall 1.0).
await brain.find({
query: 'quarterly revenue anomalies',
where: { region: 'EMEA', year: 2026 }, // resolved to an exact bitmap FIRST
})
// the vector walk only ever visits entities that already match —
// selective filters get MORE accurate, not lessCor pushes the exact filter into the vector search; everyone else filters after.
Feature tour
Adaptive DiskANN — it tunes itself to your machine
A 100% pure-Rust implementation of the DiskANN algorithm (Microsoft Research, NeurIPS 2019: Vamana graphs + product quantization + an mmap-native format). Zero knobs, by design:
const brain = new Brainy({ storage: { type: 'filesystem', path: './data' } })
await brain.init()
// that's the entire configuration — on every machine, at every scaleIt observes available RAM and picks in-memory / compressed / on-disk operation, widens its search with corpus size, shares the box fairly with sibling instances, and grows its id space to ~51B entities. Same binary, same files — from a laptop to a 128 GB server, it simply uses what it finds.
How Adaptive DiskANN works · ADR-002: why 100% Rust
Streaming inserts that never freeze
Writes absorb instantly and flush to small immutable segments in O(new-data) — seconds at any corpus size, not hours of reindexing. Index maintenance runs off-thread; consolidation is rare, background, and never blocks a read or a write.
// keep writing at any corpus size — reads stay live throughout
for (const doc of firehose) await brain.add(doc)
await brain.find({ query: 'still answering' }) // never waits on a rebuildA brain that grows fast never hits a rebuild wall.
Time travel
Point-in-time queries across vectors, graph, and metadata together — plus speculative transactions. No other vector database ships this.
const lastWeek = await brain.asOf(Date.now() - 7 * 86_400_000)
await lastWeek.find({ query: 'what did we know then?' }) // full query surface, past stateCold start = warm
Every index is mmap-durable — the files are the state. A restart serves identical results from the first query, with no rebuild and no warm-up phase (measured: cold-reopen recall matches warm exactly).
Local embeddings, embedded engine
An on-box ML runtime (Candle) — no per-document API fees, no data egress, fully offline-capable. And the whole thing is a library, not a database server: nothing to deploy, monitor, or operate.
Automatic upgrades
Opening an older-format brain triggers a coordinated, observable migration: progress reporting, an automatic pre-upgrade backup, and a guarantee it never serves half-built state or loses a write.
Upgrading from 2.x / brainy 7.x
One box, hundreds of brains — or one brain at billion scale
The same zero-config design serves both extremes. Per-user isolation: run one brain per user or customer — each a physically separate database (own files, own indexes; no shared store, no cross-tenant query surface to secure). Every brain sizes itself from observed RAM ÷ active brains, a resource manager rebalances budgets at runtime, and mmap paging means idle tenants cost almost nothing — hundreds of isolated brains fit one commodity box. Or scale one up: the identical engine takes a single brain to billions of entities on that same box. No sharding tier, no "multi-tenant edition" — one design, both shapes.
Scaling: sizing, tenants, limits
Measured, not promised
SIFT benchmarks on commodity-class hardware; reproducible via scripts/verify-*.mjs; methodology + honesty notes in docs/comparison.md.
| Scale | Recall@10 | Median latency | |---|---|---| | 1M | 0.9942 | 0.31 ms | | 10M | 0.9647 | 1.33 ms | | 100M | 0.9559 | ~7 ms — one box, at the published DiskANN billion-scale operating point | | 1B | in validation | measured run in progress; labeled projected until it lands |
Cold-reopen recall matches warm exactly (measured). The write path holds flat RSS from 100k → 1M entities (measured). Billion-scale claims stay labeled until the 1B run completes — that's the house rule.
Built for teams that can't send data anywhere
Everything runs on your hardware, in your process: search, storage, and embedding generation (the on-box ML runtime means documents never leave the machine, even to be vectorized). Air-gapped deployments work — license keys verify offline, and the daily license ping (key id, machine fingerprint, size bucket; never data) degrades gracefully when there's no route out. Point-in-time queries give you an audit dimension most databases can't: what did the system know, and when?
Licensing that never breaks your app
Start free on brainy. When you outgrow it, npm install @soulcraft/cor — nothing else changes. And in the other direction: no key — or an expired one, or no network — means cor quietly steps aside and you're running open-source brainy on the same files. There is no kill switch, and we will never build one. Pricing follows the size of your brain, not seats or nodes: free under 1M entities (that's brainy), then simple tiers as you grow. Once a day cor reports its key id, a machine fingerprint, and a coarse size bucket — never data, queries, or content. See soulcraft.com/pricing.
Requirements
- Node.js ≥ 22 (Bun ≥ 1.1 as a runtime) · TypeScript-native, ESM
@soulcraft/brainy≥ 8.0 · local filesystem storage (mmap requires real files)- 32 GB RAM serves ~1B entities; 64 GB recommended — see docs/scaling.md
Learn more
| Topic | Doc | |---|---| | How Adaptive DiskANN works | docs/diskann.md · ADR-002: why 100% Rust | | Benchmarks + methodology | docs/performance.md · docs/comparison.md · docs/verification-report.md | | Scaling: sizing, tenants, limits | docs/scaling.md · docs/billion-scale.md · docs/deployment-limits.md | | Durability + snapshots | docs/snapshot-safety.md | | Time travel | ADR-003: semantic time travel | | Upgrading from 2.x / brainy 7.x | docs/migration-3.0.md |
The open-core promise
Every accelerated path has a working JavaScript default in brainy (MIT). You upgrade for speed, not capability — and you can downgrade at any time with zero migration, because cor works on brainy's files, not its own. Your data is never hostage to a subscription.
