winnow-db

v0.1.0

Published

5 months ago

High-performance vector database for JavaScript. Runs in browser & Node.js. No server required.

0High
0Medium
0Low

vector-database vector-db embeddings semantic-search similarity-search hnsw wasm webassembly rust ai machine-learning rag llm offline

WinnowDB

High-performance vector database for JavaScript/TypeScript

Run semantic search, recommendations, and AI applications entirely in the browser or Node.js. No server required.

Quick Install

npm install @winnow-db/winnow-db

30-Second Example

import init, { WinnowCollection } from '@winnow-db/winnow-db';

// Initialize (call once)
await init();

// Create collection
const db = new WinnowCollection("vectors", {
    dim: 128,
    index_type: "HNSW",
    max_elements: 10000
}, null);

// Add vectors
db.add(1, new Float32Array(128).fill(0.5), null, null, '{"name": "doc1"}');
db.add(2, new Float32Array(128).fill(0.3), null, null, '{"name": "doc2"}');

// Search (returns [[id, distance], ...])
const results = db.search(new Float32Array(128).fill(0.5), 5, null, null);
console.log(results); // [[1, 0], [2, 0.04], ...]

// Cleanup
db.free();

API Reference

Create Collection

const collection = new WinnowCollection(name, config, null);

Config options:

| Option | Type | Default | Description | |--------|------|---------|-------------| | dim | number | required | Vector dimension | | index_type | string | "HNSW" | Index type: "HNSW", "IVFPQ", "SQ", "BQ" | | max_elements | number | required | Maximum vectors | | m | number | 16 | HNSW connections per layer | | ef_construction | number | 100 | HNSW build quality |

Add Vector

collection.add(id, vector, null, null, payload);

id: number - Unique ID
vector: Float32Array - Your embedding
payload: string - JSON metadata (optional)

Search

const results = collection.search(queryVector, k, filter, null);
// Returns: [[id, distance], [id, distance], ...]

queryVector: Float32Array - Query embedding
k: number - How many results
filter: object - Filter by payload fields (optional)

Other Methods

collection.delete(id)           // Remove vector
collection.exists(id)           // Check if exists
collection.count()              // Get count
collection.get_payload(id)      // Get metadata
collection.update_payload(id, json) // Update metadata
collection.clear()              // Remove all
collection.snapshot()           // Export as bytes
collection.free()               // Cleanup memory

Examples

Semantic Text Search

import init, { WinnowCollection } from '@winnow-db/winnow-db';

await init();

const db = new WinnowCollection("docs", {
    dim: 384, // all-MiniLM-L6-v2 dimension
    index_type: "HNSW",
    max_elements: 10000
}, null);

// Add documents (use your embedding model)
const docs = [
    { id: 1, text: "JavaScript is a programming language" },
    { id: 2, text: "Python is great for data science" },
    { id: 3, text: "Rust is fast and memory-safe" }
];

for (const doc of docs) {
    const embedding = await getEmbedding(doc.text); // Your embedding function
    db.add(doc.id, embedding, null, null, JSON.stringify(doc));
}

// Search
const query = await getEmbedding("web development language");
const results = db.search(query, 3, null, null);

for (const [id, score] of results) {
    console.log(db.get_payload(id));
}

Filtered Search

// Add with category
db.add(1, vec1, null, null, '{"title": "JS Guide", "category": "web"}');
db.add(2, vec2, null, null, '{"title": "ML Intro", "category": "ai"}');
db.add(3, vec3, null, null, '{"title": "React Tips", "category": "web"}');

// Search only "web" category
const results = db.search(query, 10, { category: "web" }, null);

Save and Load

// Save
const bytes = db.snapshot();
localStorage.setItem('mydb', btoa(String.fromCharCode(...bytes)));

// Load
const saved = localStorage.getItem('mydb');
const bytes = new Uint8Array([...atob(saved)].map(c => c.charCodeAt(0)));
const db = WinnowCollection.load_snapshot(bytes, null);

Browser Usage

<!DOCTYPE html>
<html>
<head>
    <title>WinnowDB Demo</title>
</head>
<body>
    <script type="module">
        import init, { WinnowCollection } from './node_modules/@winnow-db/winnow-db/winnow_db.js';
        
        await init();
        
        const db = new WinnowCollection("search", {
            dim: 128,
            index_type: "HNSW", 
            max_elements: 1000
        }, null);
        
        // Add some vectors
        for (let i = 0; i < 100; i++) {
            const vec = new Float32Array(128).fill(i / 100);
            db.add(i, vec, null, null, JSON.stringify({ id: i }));
        }
        
        // Search
        const query = new Float32Array(128).fill(0.5);
        const results = db.search(query, 5, null, null);
        console.log('Results:', results);
    </script>
</body>
</html>

Performance

| Vectors | Insert | Search | Latency | |---------|--------|--------|---------| | 1,000 | 4,300/s | 20,000 qps | 0.05ms | | 10,000 | 2,000/s | 13,000 qps | 0.08ms | | 100,000 | 1,000/s | 5,000 qps | 0.2ms |

Use Cases

RAG: Retrieval for LLM applications
Search: Semantic search in documents
Recommendations: Similar items/users
Image Search: Find similar images
Offline: Works without internet

License

MIT