weavatrix-search-vector

A persistent, mutable, bounded vector index — deterministic HNSW plus an exact oracle — written in Rust and exposed to Node.js and Bun through Node-API.

You bring the vectors. This package stores them, searches them, mutates them without a rebuild, filters them on metadata, and writes them to one checksummed file. It downloads no model, calls no embedding API, and opens no socket.

npm install weavatrix-search-vector
# or
bun add weavatrix-search-vector

const { VectorIndex } = require('weavatrix-search-vector')

const index = VectorIndex.build(
  [
    { key: 1, vector: Float32Array.of(0.1, 0.9, 0.3, 0.2) },
    { key: 2, vector: Float32Array.of(0.8, 0.1, 0.1, 0.5) },
  ],
  { dimensions: 4, metric: 'cosine' },
)

index.search(Float32Array.of(0.1, 0.9, 0.3, 0.2), 2)
// [ { key: 1, distance: 0 }, { key: 2, distance: 0.63… } ]

index.save('vectors.wvsv')

Three things that shape the API

Deterministic. Same vectors, same config, same seed ⇒ same graph and the same ranked answer, on every platform. Nothing depends on wall-clock time or unseeded randomness.

Exact is always available. { exact: true } compares every stored vector. It is the oracle you check the approximate path against, and it is still far faster than doing it in JavaScript.

Keys are numbers, checked. Vector keys are u64 in Rust. Here they are JavaScript numbers restricted to integers in 0 … 9007199254740991; anything outside is rejected, never silently truncated. The HNSW seed does not fit that range, so it crosses the boundary as a decimal string.

Two index types

| | VectorIndex | MutableVectorIndex | | --- | --- | --- | | Built from | a fixed set | a set you keep changing | | Insert / update / delete | — | yes, without rebuilding | | Metadata and filters | — | yes | | Batch query API | yes | — | | Persistence | .save / .load / .readMetadata | .save / .load (vectors, graphs, delta, tombstones, metadata under one checksum) |

Choose VectorIndex for a snapshot you rebuild wholesale, and MutableVectorIndex for a living store.

Input forms

Both constructors accept either shape:

// Records — convenient.
VectorIndex.build([{ key: 1, vector: Float32Array.of(…), metadata: { lang: 'rust' } }], config)

// Columnar — one flat Float32Array of keys.length * dimensions scalars.
// An embedding batch never has to be split into objects.
VectorIndex.build({ keys, vectors, metadata }, config)

vector accepts a Float32Array or any array-like of numbers; keys accepts a Float64Array or an array of numbers.

API

IndexConfig

| Field | Type | Default | Meaning | | --- | --- | --- | --- | | dimensions | number | required | Scalars in every stored vector and query. | | metric | 'cosine' \| 'dot' \| 'squaredEuclidean' | 'cosine' | cosine is one minus cosine similarity; dot is the negative inner product, so a longer vector in the same direction outranks an identical one; squaredEuclidean is non-negative. | | connectivity | number | 12 | Upper-layer link budget. Layer zero uses twice this. | | expansionBuild | number | 48 | Candidate width while constructing links. Must be ≥ connectivity. | | expansionQuery | number | 24 | Default candidate width for approximate queries. | | replicas | number | 1 | Independently seeded deterministic graphs. | | buildThreads, queryThreads | number | available parallelism, capped at 16 | Worker budgets. | | seed | number \| string | '7640891576956012809' | A u64. Pass a decimal string above 2^53 - 1. |

config() returns the resolved policy with seed always a decimal string.

class VectorIndex

| Member | Returns | Notes | | --- | --- | --- | | static build(records, config) | VectorIndex | | | static load(path) | VectorIndex | Validates structure, finite vectors, and the payload checksum. | | static readMetadata(path) | SnapshotMetadata | { formatVersion, vectorCount, serializedBytes, config } — header only, no vectors or graphs loaded. | | size | number | | | dimensions | number | | | estimatedMemoryBytes | number | Resident vector and graph storage; excludes allocator metadata and query scratch. | | config() | ResolvedIndexConfig | | | keys() | Float64Array | Ascending. | | vector(key) | Float32Array \| undefined | The normalized stored vector. | | search(query, count, policy?) | SearchHit[] | | | searchBatch(queries, count, policy?) | SearchHit[][] | Independent queries across bounded scoped workers, each reusing its heaps. Accepts an array of vectors or one flat Float32Array. | | save(path) | this | Atomic, one checksum. |

class MutableVectorIndex

| Member | Returns | Notes | | --- | --- | --- | | static build(records, config) | MutableVectorIndex | | | static load(path) | MutableVectorIndex | | | size | number | Live keys. | | stagedSize | number | Pending inserts and updates not yet sealed. | | sealedSize | number | Records folded into the sealed layer. | | deltaSize | number | Everything outside the immutable base. | | config() | ResolvedIndexConfig | | | insert(key, vector, metadata?) | this | Throws on a duplicate live key. | | upsert(key, vector, metadata?) | 'inserted' \| 'updated' | | | insertBatch(records) | this | Atomic: validates every record, then applies all or none. | | upsertBatch(records) | MutationOutcome[] | Results follow input order. | | delete(key) | boolean | Tombstone. false when the key was not live. | | deleteBatch(keys) | number | How many were actually removed. | | rename(from, to) | boolean | | | setMetadata(key, metadata?) | this | Omitting metadata clears the fields. | | metadata(key) | ReadMetadata \| undefined | undefined both for an unknown key and for a key carrying no fields. | | search(query, count) | SearchHit[] | Searches base, sealed layer, and the exact delta, then merges deterministically. | | searchFiltered(query, count, filter) | SearchHit[] | | | shouldCompact(maximumDelta) | boolean | | | sealDelta() | this | Folds the delta into an immutable sealed layer. | | compact() | this | Rebuilds one base from base, sealed layer, and delta. Readers never observe a partial rebuild. | | save(path) | this | Vectors, graphs, staged vectors, tombstones, and metadata, atomically, without forcing compaction. |

SearchPolicy

Passed per query; nothing is rebuilt.

| Field | Default | Effect | | --- | --- | --- | | expansion | config.expansionQuery | Minimum graph candidates retained per replica. Higher means more recall and more time. | | routingProbes | 1 | Deterministic routing buckets probed after graph traversal. Useful when raising expansion alone stops helping. | | exact | false | Compares every stored vector instead of traversing the graph. |

SearchHit

{ key: number, distance: number }, ascending by distance then key. Lower is better under every metric.

Metadata and filters

A metadata value is written in shorthand or tagged explicitly:

| You write | Stored as | | --- | --- | | 'rust' | { text: 'rust' } | | true | { bool: true } | | 42 (safe integer) | { i64: 42 } | | { u64: 9007199254740991 } | { u64: … } | | { bytes: Uint8Array.of(1, 2, 3) } | { bytes: … } |

metadata(key) always returns the tagged form, so a round trip is unambiguous. A non-integer number is rejected rather than coerced.

Filters compose:

index.searchFiltered(query, 10, {
  and: [
    { equal: { field: 'lang', value: 'rust' } },
    { i64Range: { field: 'lines', minimum: 100, maximum: 400 } },
    { not: { exists: 'deprecated' } },
  ],
})

| Operator | Shape | | --- | --- | | 'all' | the bare string | | { exists: field } | field is present | | { equal: { field, value } } | tag and value must match | | { i64Range: { field, minimum?, maximum? } } | inclusive, either bound optional | | { u64Range: { field, minimum?, maximum? } } | inclusive | | { textPrefix: { field, prefix } } | | | { and: [...] }, { or: [...] }, { not: … } | |

Filtering happens during traversal, with an exact fallback when a selective filter starves the graph, so a filtered query cannot silently return fewer neighbours than exist.

Errors

| code | Cause | | --- | --- | | InvalidArg | Key outside the safe-integer range, unknown metric or filter operator, wrong scalar count, unusable config, malformed metadata value. | | GenericFailure | Duplicate key on insert, dimension mismatch, zero vector under cosine, storage or checksum failure. |

What ships

| | | | --- | --- | | Runtimes | Node.js 18+ (Node-API 8), Bun 1.4+ | | Platforms | Windows x64/arm64, macOS x64/arm64, glibc Linux x64/arm64 | | Install script | none | | Network at install | none | | Runtime dependencies | none | | Platform packages | none — all six bindings are in this one tarball |

Scalar quantization, memory-mapped snapshots, multi-vector keys, KNN graph construction, and shard coordination exist in the Rust crate and are not exposed here yet.

Measured

benchmark/RESULTS.md is generated from the weavatrix-benchmarks harness. Recall is measured, not assumed: the JavaScript baseline computes the exact answer, the exact Weavatrix path must reproduce it identically before anything is timed, and every approximate row reports its overlap with it.

Medians of three independent runs over 20,000 vectors of 64 dimensions, against exact brute force in JavaScript:

| Query | Recall | Node 24 | Bun 1.3 | | --- | ---: | ---: | ---: | | Clustered — exact | 1.000 | 8.43x | 8.33x | | Clustered — default policy | 1.000 | 41.6x | 37.2x | | Clustered — expansion 128, 8 probes | 1.000 | 18.0x | 20.0x | | Clustered — batch API | 1.000 | 58.5x | 50.0x | | Uniform — exact | 1.000 | 9.40x | 7.33x | | Uniform — default policy | 0.644 | 22.5x | 19.0x | | Uniform — expansion 128, 8 probes | 0.960 | 8.70x | 7.51x | | Uniform — batch API | 0.644 | 44.9x | 39.9x |

The clustered corpus is what real embeddings look like, and there the default policy already returns the exact answer, so raising expansion only costs time. The uniform corpus is the adversarial case for any graph index — vectors in 64 dimensions that are close to mutually orthogonal — and it is published so the default policy's recall is not mistaken for a universal number. The exact path stays available in both, and still beats the JavaScript baseline.

Repository: Weavatrix/weavatrix-search-vector · Rust crate: crates.io/crates/weavatrix-search-vector · License: MIT